Spring Cloud Data Flow Server for Cloud Foundry
   Next

Spring Cloud Data Flow Server for Cloud Foundry

Authors

Sabby Anandan, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Thomas Risberg, Marius Bogoevici, Josh Long, Michael Minella

1.0.2.BUILD-SNAPSHOT

Copyright © 2013-2016 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Spring Cloud Data Flow for Cloud Foundry
1. Spring Cloud Data Flow
2. Spring Cloud Stream
3. Spring Cloud Task
II. Architecture
4. Introduction
5. Microservice Architectural Style
5.1. Comparison to other Platform architectures
6. Streaming Applications
6.1. Imperative Programming Model
6.2. Functional Programming Model
7. Streams
7.1. Topologies
7.2. Concurrency
7.3. Partitioning
7.4. Message Delivery Guarantees
8. Analytics
9. Task Applications
10. Data Flow Server
10.1. Endpoints
10.2. Customization
10.3. Security
11. Runtime
11.1. Fault Tolerance
11.2. Resource Management
11.3. Scaling at runtime
11.4. Application Versioning
III. Getting started
12. Deploying on Cloud Foundry
12.1. Provision a Redis service instance on Cloud Foundry
12.2. Provision a Rabbit service instance on Cloud Foundry
12.3. Provision a MySQL service instance on Cloud Foundry
12.4. Download the Spring Cloud Data Flow Server and Shell apps
12.5. Running the Server
12.5.1. Deploying and Running the Server app on Cloud Foundry
Configuring Defaults for Deployed Apps
12.5.2. Running the Server app locally
12.6. Running Tasks
12.7. Running Spring Cloud Data Flow Shell locally
13. Security
14. Application Names and Prefixes
15. Authentication and Cloud Foundry
16. Configuration Reference
16.1. Using Spring Cloud Config Server
17. Application Level Service Bindings
18. A Note About User Provided Services
19. Application Rolling Upgrades
IV. Streams
20. Introduction
21. Stream DSL
22. Register a Stream App
22.1. Whitelisting application properties
23. Creating a Stream
23.1. Application properties
23.1.1. Passing application properties when creating a stream
23.1.2. Passing application properties when deploying a stream
23.1.3. Passing stream partition properties during stream deployment
23.1.4. Overriding application properties during stream deployment
23.2. Deployment properties
23.2.1. Passing instance count as deployment property
23.2.2. Inline vs file reference properties
24. Destroying a Stream
25. Deploying and Undeploying Streams
26. Other Source and Sink Application Types
27. Simple Stream Processing
28. Stateful Stream Processing
29. Tap a Stream
30. Using Labels in a Stream
31. Explicit Broker Destinations in a Stream
32. Directed Graphs in a Stream
32.1. Common application properties
33. Stream applications with multiple binder configurations
V. Tasks
34. Introducing Spring Cloud Task
35. The Lifecycle of a task
35.1. Registering a Task Application
35.2. Creating a Task
35.3. Launching a Task
35.4. Reviewing Task Executions
35.5. Destroying a Task
36. Task Repository
36.1. Configuring the Task Execution Repository
36.1.1. Local
36.2. Datasource
37. Subscribing to Task/Batch Events
38. Launching Tasks from a Stream
38.1. TriggerTask
38.2. Translator
VI. Tasks on Cloud Foundry
39. Version Compatibility
40. Tooling
41. Running Task Applications
41.1. Create a Task
41.2. Launch a Task
41.3. View Task Logs
41.4. List Tasks
41.5. List Task Executions
41.6. Destroy a Task
VII. Dashboard
42. Introduction
43. Apps
44. Runtime
45. Streams
46. Create Stream
47. Tasks
47.1. Apps
47.1.1. Create a Task Definition from a selected Task App
47.1.2. View Task App Details
47.2. Definitions
47.2.1. Launching Tasks
47.3. Executions
48. Jobs
48.1. List job executions
48.1.1. Job execution details
48.1.2. Step execution details
48.1.3. Step Execution Progress
49. Analytics
VIII. ‘How-to’ guides
50. Configure Maven Properties
IX. Appendices
A. Migrating from Spring XD to Spring Cloud Data Flow
A.1. Terminology Changes
A.2. Modules to Applications
A.2.1. Custom Applications
A.2.2. Application Registration
A.2.3. Application Properties
A.3. Message Bus to Binders
A.3.1. Message Bus
A.3.2. Binders
A.3.3. Named Channels
A.3.4. Directed Graphs
A.4. Batch to Tasks
A.5. Shell/DSL Commands
A.6. REST-API
A.7. UI / Flo
A.8. Architecture Components
A.8.1. ZooKeeper
A.8.2. RDBMS
A.8.3. Redis
A.8.4. Cluster Topology
A.9. Central Configuration
A.10. Distribution
A.11. Hadoop Distribution Compatibility
A.12. YARN Deployment
A.13. Use Case Comparison
A.13.1. Use Case #1
A.13.2. Use Case #2
A.13.3. Use Case #3
B. Building
B.1. Basic Compile and Test
B.2. Documentation
B.3. Working with the code
B.3.1. Importing into eclipse with m2eclipse
B.3.2. Importing into eclipse without m2eclipse
   Next
   Part I. Spring Cloud Data Flow for Cloud Foundry