Spring Cloud Data Flow Reference Guide
   Next

Spring Cloud Data Flow Reference Guide

Authors

Sabby Anandan, Marius Bogoevici, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Patrick Peralta, Glenn Renfro, Thomas Risberg, Dave Syer, David Turanski, Janne Valkealahti

1.0.0.RELEASE

Copyright © 2013-2016 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Preface
1. About the documentation
2. Getting help
II. Spring Cloud Data Flow Overview
3. Introducing Spring Cloud Data Flow
3.1. Features
III. Architecture
4. Introduction
5. Microservice Architectural Style
5.1. Comparison to other Platform architectures
6. Streaming Applications
6.1. Imperative Programming Model
6.2. Functional Programming Model
7. Streams
7.1. Topologies
7.2. Concurrency
7.3. Partitioning
7.4. Message Delivery Guarantees
8. Analytics
9. Task Applications
10. Data Flow Server
10.1. Endpoints
10.2. Customization
10.3. Security
11. Runtime
11.1. Fault Tolerance
11.2. Resource Management
11.3. Scaling at runtime
11.4. Application Versioning
IV. Getting started
12. System Requirements
13. Controlling features with Data Flow server
14. Deploying Spring Cloud Data Flow
14.1. Deploying 'local'
15. RDBMS configuration
16. Security
16.1. Enabling HTTPS
16.1.1. Using Self-Signed Certificates
16.2. Enabling Authentication
16.3. Authentication using the Spring Cloud Data Flow Shell
16.4. Authentication Examples
16.4.1. Local OAuth2 Server
16.4.2. Authentication using GitHub
V. Streams
17. Introduction
18. Stream DSL
19. Register a Stream App
19.1. Whitelisting application properties
20. Creating a Stream
20.1. Application properties
20.1.1. Passing application properties when creating a stream
20.1.2. Passing application properties when deploying a stream
20.1.3. Passing stream partition properties during stream deployment
20.1.4. Overriding application properties during stream deployment
20.2. Deployment properties
20.2.1. Passing instance count as deployment property
20.2.2. Inline vs file reference properties
21. Destroying a Stream
22. Deploying and Undeploying Streams
23. Other Source and Sink Application Types
24. Simple Stream Processing
25. Stateful Stream Processing
26. Tap a Stream
27. Using Labels in a Stream
28. Explicit Broker Destinations in a Stream
29. Directed Graphs in a Stream
29.1. Common application properties
VI. Tasks
30. Introducing Spring Cloud Task
31. The Lifecycle of a task
31.1. Registering a Task Application
31.2. Creating a Task
31.3. Launching a Task
31.4. Reviewing Task Executions
31.5. Destroying a Task
32. Task Repository
32.1. Configuring the Task Execution Repository
32.1.1. Local
32.2. Datasource
33. Subscribing to Task/Batch Events
34. Launching Tasks from a Stream
34.1. TriggerTask
34.2. Translator
VII. Dashboard
35. Introduction
36. Apps
37. Runtime
38. Streams
39. Create Stream
40. Tasks
40.1. Apps
40.1.1. Create a Task Definition from a selected Task App
40.1.2. View Task App Details
40.2. Definitions
40.2.1. Launching Tasks
40.3. Executions
41. Jobs
41.1. List job executions
41.1.1. Job execution details
41.1.2. Step execution details
41.1.3. Step Execution Progress
42. Analytics
VIII. ‘How-to’ guides
43. Configure Maven Properties
IX. Appendices
A. Migrating from Spring XD to Spring Cloud Data Flow
A.1. Terminology Changes
A.2. Modules to Applications
A.2.1. Custom Applications
A.2.2. Application Registration
A.2.3. Application Properties
A.3. Message Bus to Binders
A.3.1. Message Bus
A.3.2. Binders
A.3.3. Named Channels
A.3.4. Directed Graphs
A.4. Batch to Tasks
A.5. Shell/DSL Commands
A.6. REST-API
A.7. UI / Flo
A.8. Architecture Components
A.8.1. ZooKeeper
A.8.2. RDBMS
A.8.3. Redis
A.8.4. Cluster Topology
A.9. Central Configuration
A.10. Distribution
A.11. Hadoop Distribution Compatibility
A.12. YARN Deployment
A.13. Use Case Comparison
A.13.1. Use Case #1
A.13.2. Use Case #2
A.13.3. Use Case #3
B. Building
B.1. Documentation
B.2. Working with the code
B.2.1. Importing into eclipse with m2eclipse
B.2.2. Importing into eclipse without m2eclipse
C. Contributing
C.1. Sign the Contributor License Agreement
C.2. Code Conventions and Housekeeping
   Next
   Part I. Preface