Spring Cloud Data Flow Server for Kubernetes
   Next

Spring Cloud Data Flow Server for Kubernetes

Sabby Anandan, Marius Bogoevici, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Patrick Peralta, Glenn Renfro, Thomas Risberg, Florian Rosenberg, Dave Syer, David Turanski, Janne Valkealahti, Oleg Zhurakousky

1.2.2.RELEASE

Copyright © 2013-2017 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Getting Started
1. Deploying the Spring Cloud Data Flow server on Kubernetes
2. Creating and Running Streams on Kubernetes
2.1. Accessing app from outside the cluster
3. Creating and Launching Tasks on Kubernetes
4. Application Configuration
4.1. Memory and CPU Settings
4.2. Environment Variables
4.3. Liveness and Readiness Probes
II. Applications
III. Architecture
5. Introduction
6. Microservice Architectural Style
6.1. Comparison to other Platform architectures
7. Streaming Applications
7.1. Imperative Programming Model
7.2. Functional Programming Model
8. Streams
8.1. Topologies
8.2. Concurrency
8.3. Partitioning
8.4. Message Delivery Guarantees
9. Analytics
10. Task Applications
11. Data Flow Server
11.1. Endpoints
11.2. Customization
11.3. Security
12. Runtime
12.1. Fault Tolerance
12.2. Resource Management
12.3. Scaling at runtime
12.4. Application Versioning
IV. Server Configuration
13. Feature Toggles
14. General Configuration
14.1. Using ConfigMap and Secrets
15. Database Configuration
16. Security
17. Spring Cloud Deployer for Kubernetes Properties
17.1. Using Deployments
17.2. CPU and Memory Limits
17.3. Liveness and Rediness Probes Configurations
17.4. Using SPRING_APPLICATION_JSON
18. Monitoring and Management
18.1. Inspecting Server Logs
18.2. Streams
18.3. Tasks
V. Shell
19. Shell Options
20. Listing available commands
21. Tab Completion
22. White space and quote rules
22.1. Quotes and Escaping
22.1.1. Shell rules
22.1.2. DSL parsing rules
22.1.3. SpEL syntax and SpEL literals
22.1.4. Putting it all together
VI. Streams
23. Introduction
23.1. Stream Pipeline DSL
23.2. Application properties
24. Lifecycle of Streams
24.1. Register a Stream App
24.1.1. Whitelisting application properties
24.1.2. Creating and using a dedicated metadata artifact
24.1.3. Using the companion artifact
24.2. Creating custom applications
24.3. Creating a Stream
24.3.1. Application properties
Passing application properties when creating a stream
24.3.2. Deployment properties
Application properties versus Deployer properties
Passing instance count as deployment property
Inline vs file reference properties
Passing application properties when deploying a stream
Passing Spring Cloud Stream properties for the application
Passing per-binding producer consumer properties
Passing stream partition properties during stream deployment
Passing application content type properties
Overriding application properties during stream deployment
24.3.3. Common application properties
24.4. Destroying a Stream
24.5. Deploying and Undeploying Streams
25. Stream DSL
25.1. Tap a Stream
25.2. Using Labels in a Stream
25.3. Named Destinations
25.4. Fan-in and Fan-out
26. Stream applications with multiple binder configurations
27. Examples
27.1. Simple Stream Processing
27.2. Stateful Stream Processing
27.3. Other Source and Sink Application Types
VII. Tasks
28. Introduction
29. The Lifecycle of a Task
29.1. Creating a Task Application
29.1.1. Task Database Configuration
29.2. Registering a Task Application
29.3. Creating a Task Definition
29.4. Launching a Task
29.4.1. Common application properties
29.5. Reviewing Task Executions
29.6. Destroying a Task Definition
30. Subscribing to Task/Batch Events
31. Launching Tasks from a Stream
31.1. TriggerTask
31.2. TaskLaunchRequest-transform
32. Composed Tasks
32.1. Configuring the Composed Task Runner
32.1.1. Registering the Composed Task Runner
32.1.2. Configuring the Composed Task Runner
32.2. The Lifecycle of a Composed Task
32.2.1. Creating a Composed Task
Task Application Parameters
32.2.2. Launching a Composed Task
Exit Statuses
32.2.3. Destroying a Composed Task
32.2.4. Stopping a Composed Task
32.2.5. Restarting a Composed Task
33. Composed Tasks DSL
33.1. Conditional Execution
33.2. Transitional Execution
33.2.1. Basic Transition
33.2.2. Transition With a Wildcard
33.2.3. Transition With a Following Conditional Execution
33.3. Split Execution
33.3.1. Split Containing Conditional Execution
VIII. Dashboard
34. Introduction
35. Apps
35.1. Bulk Import of Applications
36. Runtime
37. Streams
38. Create Stream
39. Tasks
39.1. Apps
39.1.1. Create a Task Definition from a selected Task App
39.1.2. View Task App Details
39.2. Definitions
39.2.1. Creating Task Definitions using the bulk define interface
39.2.2. Creating Composed Task Definitions
39.2.3. Launching Tasks
39.3. Executions
40. Jobs
40.1. List job executions
40.1.1. Job execution details
40.1.2. Step execution details
40.1.3. Step Execution Progress
41. Analytics
IX. REST API Guide
X. Appendices
A. ‘How-to’ guides
A.1. Logging
A.1.1. Deployment Logs
A.1.2. Application Logs
B. Data Flow Template
B.1. Using the Data Flow Template
C. Spring XD to SCDF
C.1. Terminology Changes
C.2. Modules to Applications
C.2.1. Custom Applications
C.2.2. Application Registration
C.2.3. Application Properties
C.3. Message Bus to Binders
C.3.1. Message Bus
C.3.2. Binders
C.3.3. Named Channels
C.3.4. Directed Graphs
C.4. Batch to Tasks
C.5. Shell/DSL Commands
C.6. REST-API
C.7. UI / Flo
C.8. Architecture Components
C.8.1. ZooKeeper
C.8.2. RDBMS
C.8.3. Redis
C.8.4. Cluster Topology
C.9. Central Configuration
C.10. Distribution
C.11. Hadoop Distribution Compatibility
C.12. YARN Deployment
C.13. Use Case Comparison
C.13.1. Use Case #1
C.13.2. Use Case #2
C.13.3. Use Case #3
D. Building
D.1. Documentation
D.2. Working with the code
D.2.1. Importing into eclipse with m2eclipse
D.2.2. Importing into eclipse without m2eclipse
E. Contributing
E.1. Sign the Contributor License Agreement
E.2. Code Conventions and Housekeeping
   Next
   Part I. Getting Started