Spring Cloud Data Flow Server for Apache Mesos
   Next

Spring Cloud Data Flow Server for Apache Mesos

Sabby Anandan, Artem Bilan, Marius Bogoevici, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Patrick Peralta, Glenn Renfro, Gary Russell, Thomas Risberg, David Turanski, Janne Valkealahti

1.0.0.RELEASE

Copyright © 2013-2016 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Introduction
1. Introducing Spring Cloud Data Flow Server for Apache Mesos
2. Spring Cloud Data Flow
3. Spring Cloud Stream
4. Spring Cloud Task
II. Architecture
5. Introduction
6. Microservice Architectural Style
6.1. Comparison to other Platform architectures
7. Streaming Applications
7.1. Imperative Programming Model
7.2. Functional Programming Model
8. Streams
8.1. Topologies
8.2. Concurrency
8.3. Partitioning
8.4. Message Delivery Guarantees
9. Analytics
10. Task Applications
11. Data Flow Server
11.1. Endpoints
11.2. Customization
11.3. Security
12. Runtime
12.1. Fault Tolerance
12.2. Resource Management
12.3. Scaling at runtime
12.4. Application Versioning
III. Getting Started
13. Deploying Streams and Tasks on Apache Mesos and Marathon/Chronos
IV. Streams
14. Introduction
15. Stream DSL
16. Register a Stream App
16.1. Whitelisting application properties
17. Creating a Stream
17.1. Application properties
17.1.1. Passing application properties when creating a stream
17.1.2. Passing application properties when deploying a stream
17.1.3. Passing stream partition properties during stream deployment
17.1.4. Overriding application properties during stream deployment
17.2. Deployment properties
17.2.1. Passing instance count as deployment property
17.2.2. Inline vs file reference properties
18. Destroying a Stream
19. Deploying and Undeploying Streams
20. Other Source and Sink Application Types
21. Simple Stream Processing
22. Stateful Stream Processing
23. Tap a Stream
24. Using Labels in a Stream
25. Explicit Broker Destinations in a Stream
26. Directed Graphs in a Stream
26.1. Common application properties
27. Stream applications with multiple binder configurations
V. Tasks
28. Introducing Spring Cloud Task
29. The Lifecycle of a task
29.1. Registering a Task Application
29.2. Creating a Task
29.3. Launching a Task
29.4. Reviewing Task Executions
29.5. Destroying a Task
30. Task Repository
30.1. Configuring the Task Execution Repository
30.1.1. Local
30.2. Datasource
31. Subscribing to Task/Batch Events
32. Launching Tasks from a Stream
32.1. TriggerTask
32.2. Translator
VI. Dashboard
33. Introduction
34. Apps
35. Runtime
36. Streams
37. Create Stream
38. Tasks
38.1. Apps
38.1.1. Create a Task Definition from a selected Task App
38.1.2. View Task App Details
38.2. Definitions
38.2.1. Launching Tasks
38.3. Executions
39. Jobs
39.1. List job executions
39.1.1. Job execution details
39.1.2. Step execution details
39.1.3. Step Execution Progress
40. Analytics
VII. Appendices
A. Migrating from Spring XD to Spring Cloud Data Flow
A.1. Terminology Changes
A.2. Modules to Applications
A.2.1. Custom Applications
A.2.2. Application Registration
A.2.3. Application Properties
A.3. Message Bus to Binders
A.3.1. Message Bus
A.3.2. Binders
A.3.3. Named Channels
A.3.4. Directed Graphs
A.4. Batch to Tasks
A.5. Shell/DSL Commands
A.6. REST-API
A.7. UI / Flo
A.8. Architecture Components
A.8.1. ZooKeeper
A.8.2. RDBMS
A.8.3. Redis
A.8.4. Cluster Topology
A.9. Central Configuration
A.10. Distribution
A.11. Hadoop Distribution Compatibility
A.12. YARN Deployment
A.13. Use Case Comparison
A.13.1. Use Case #1
A.13.2. Use Case #2
A.13.3. Use Case #3
B. Test Cluster
B.1. Clone and Download files
B.2. Start Vagrant VMs for the Mesos cluster
B.3. Install Chronos as a Service running in Marathon
C. Building
C.1. Documentation
C.2. Working with the code
C.2.1. Importing into eclipse with m2eclipse
C.2.2. Importing into eclipse without m2eclipse
D. Contributing
D.1. Sign the Contributor License Agreement
D.2. Code Conventions and Housekeeping
   Next
   Part I. Introduction