Spring Cloud Data Flow Server for Cloud Foundry
   Next

Spring Cloud Data Flow Server for Cloud Foundry

Authors

Sabby Anandan, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Thomas Risberg, Marius Bogoevici, Josh Long, Michael Minella

1.1.1.RELEASE

Copyright © 2013-2016 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Spring Cloud Data Flow for Cloud Foundry
1. Spring Cloud Data Flow
2. Spring Cloud Stream
3. Spring Cloud Task
II. Architecture
4. Introduction
5. Microservice Architectural Style
5.1. Comparison to other Platform architectures
6. Streaming Applications
6.1. Imperative Programming Model
6.2. Functional Programming Model
7. Streams
7.1. Topologies
7.2. Concurrency
7.3. Partitioning
7.4. Message Delivery Guarantees
8. Analytics
9. Task Applications
10. Data Flow Server
10.1. Endpoints
10.2. Customization
10.3. Security
11. Runtime
11.1. Fault Tolerance
11.2. Resource Management
11.3. Scaling at runtime
11.4. Application Versioning
III. Getting started
12. Deploying on Cloud Foundry
12.1. Provision a Redis service instance on Cloud Foundry
12.2. Provision a Rabbit service instance on Cloud Foundry
12.3. Provision a MySQL service instance on Cloud Foundry
12.4. Download the Spring Cloud Data Flow Server and Shell apps
12.5. Running the Server
12.5.1. Deploying and Running the Server app on Cloud Foundry
Configuring Defaults for Deployed Apps
12.5.2. Running the Server app locally
12.5.3. Sample Manifest Template
12.6. Running Spring Cloud Data Flow Shell locally
13. Security
13.1. Authentication and Cloud Foundry
14. Application Names and Prefixes
14.1. Using Custom Routes
15. Configuration Reference
15.1. Understanding what’s going on
15.2. Using Spring Cloud Config Server
15.2.1. Spring Cloud Data Flow and Spring Cloud Config Server
15.2.2. Stream, Task, and Spring Cloud Config Server
15.2.3. Sample Manifest Template
15.2.4. Self-signed SSL Certificate and Spring Cloud Config Server
16. Application Level Service Bindings
17. A Note About User Provided Services
18. Application Rolling Upgrades
19. Maximum Disk Quota Configuration
19.1. PCF’s Operations Manager Configuration
19.2. Scale Application
19.3. Configuring target free disk percentage
IV. Server Configuration
20. Feature Toggles
21. Database Configuration
22. Security
22.1. Enabling HTTPS
22.1.1. Using Self-Signed Certificates
22.1.2. Self-Signed Certificates and the Shell
22.2. Basic Authentication
22.2.1. File based authentication
22.2.2. LDAP Authentication
LDAP Transport Security
22.3. OAuth 2.0
22.3.1. Authentication using the Spring Cloud Data Flow Shell
22.3.2. OAuth2 Authentication Examples
Local OAuth2 Server
Authentication using GitHub
22.4. Securing the Spring Boot Management Endpoints
23. Monitoring and Management
23.1. Spring Boot Admin
23.2. Monitoring Deployed Applications
V. Streams
24. Introduction
25. Stream DSL
26. Register a Stream App
26.1. Whitelisting application properties
27. Creating custom applications
28. Creating a Stream
28.1. Application properties
28.1.1. Passing application properties when creating a stream
28.2. Deployment properties
28.2.1. Passing instance count as deployment property
28.2.2. Inline vs file reference properties
28.2.3. Passing application properties when deploying a stream
28.2.4. Passing Spring Cloud Stream properties for the application
28.2.5. Passing per-binding producer consumer properties
28.2.6. Passing stream partition properties during stream deployment
28.2.7. Passing application content type properties
28.2.8. Overriding application properties during stream deployment
28.3. Deployment properties
28.3.1. Passing instance count as deployment property
28.3.2. Inline vs file reference properties
29. Destroying a Stream
30. Deploying and Undeploying Streams
31. Other Source and Sink Application Types
32. Simple Stream Processing
33. Stateful Stream Processing
34. Tap a Stream
35. Using Labels in a Stream
36. Explicit Broker Destinations in a Stream
37. Directed Graphs in a Stream
37.1. Common application properties
38. Stream applications with multiple binder configurations
VI. Tasks
39. Introducing Spring Cloud Task
40. The Lifecycle of a task
40.1. Creating a custom Task Application
40.2. Registering a Task Application
40.3. Creating a Task
40.4. Launching a Task
40.5. Reviewing Task Executions
40.6. Destroying a Task
41. Task Repository
41.1. Configuring the Task Execution Repository
41.1.1. Local
41.1.2. Task Application Repository
41.2. Datasource
42. Subscribing to Task/Batch Events
43. Launching Tasks from a Stream
43.1. TriggerTask
43.2. Translator
VII. Tasks on Cloud Foundry
44. Version Compatibility
45. Tooling
46. Task Database Schema
47. Running Task Applications
47.1. Create a Task
47.2. Launch a Task
47.3. View Task Logs
47.4. List Tasks
47.5. List Task Executions
47.6. Destroy a Task
47.7. Deleting Task From Cloud Foundry
VIII. Dashboard
48. Introduction
49. Apps
49.1. Bulk Import of Applications
50. Runtime
51. Streams
52. Create Stream
53. Tasks
53.1. Apps
53.1.1. Create a Task Definition from a selected Task App
53.1.2. View Task App Details
53.2. Definitions
53.2.1. Creating Task Definitions using the bulk define interface
53.2.2. Launching Tasks
53.3. Executions
54. Jobs
54.1. List job executions
54.1.1. Job execution details
54.1.2. Step execution details
54.1.3. Step Execution Progress
55. Analytics
IX. ‘How-to’ guides
56. Configure Maven Properties
57. Logging
57.1. Deployment Logs
57.2. Application Logs
X. Appendices
A. Migrating from Spring XD to Spring Cloud Data Flow
A.1. Terminology Changes
A.2. Modules to Applications
A.2.1. Custom Applications
A.2.2. Application Registration
A.2.3. Application Properties
A.3. Message Bus to Binders
A.3.1. Message Bus
A.3.2. Binders
A.3.3. Named Channels
A.3.4. Directed Graphs
A.4. Batch to Tasks
A.5. Shell/DSL Commands
A.6. REST-API
A.7. UI / Flo
A.8. Architecture Components
A.8.1. ZooKeeper
A.8.2. RDBMS
A.8.3. Redis
A.8.4. Cluster Topology
A.9. Central Configuration
A.10. Distribution
A.11. Hadoop Distribution Compatibility
A.12. YARN Deployment
A.13. Use Case Comparison
A.13.1. Use Case #1
A.13.2. Use Case #2
A.13.3. Use Case #3
B. Building
B.1. Basic Compile and Test
B.2. Documentation
B.3. Working with the code
B.3.1. Importing into eclipse with m2eclipse
B.3.2. Importing into eclipse without m2eclipse
   Next
   Part I. Spring Cloud Data Flow for Cloud Foundry