Spring Cloud Data Flow Server for Cloud Foundry
   Next

Spring Cloud Data Flow Server for Cloud Foundry

Authors

Sabby Anandan, Eric Bottard, Mark Fisher, Ilayaperumal Gopinathan, Gunnar Hillert, Mark Pollack, Thomas Risberg, Marius Bogoevici, Josh Long, Michael Minella

1.2.0.M2

Copyright © 2013-2016 Pivotal Software, Inc.

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

Table of Contents

I. Spring Cloud Data Flow for Cloud Foundry
1. Spring Cloud Data Flow
2. Spring Cloud Stream
3. Spring Cloud Task
II. Architecture
4. Introduction
5. Microservice Architectural Style
5.1. Comparison to other Platform architectures
6. Streaming Applications
6.1. Imperative Programming Model
6.2. Functional Programming Model
7. Streams
7.1. Topologies
7.2. Concurrency
7.3. Partitioning
7.4. Message Delivery Guarantees
8. Analytics
9. Task Applications
10. Data Flow Server
10.1. Endpoints
10.2. Customization
10.3. Security
11. Runtime
11.1. Fault Tolerance
11.2. Resource Management
11.3. Scaling at runtime
11.4. Application Versioning
III. Getting started
12. Deploying on Cloud Foundry
12.1. Provision a Redis service instance on Cloud Foundry
12.2. Provision a Rabbit service instance on Cloud Foundry
12.3. Provision a MySQL service instance on Cloud Foundry
12.4. Download the Spring Cloud Data Flow Server and Shell apps
12.5. Running the Server
12.5.1. Deploying and Running the Server app on Cloud Foundry
Configuring Defaults for Deployed Apps
12.5.2. Running the Server app locally
12.5.3. Sample Manifest Template
12.6. Running Spring Cloud Data Flow Shell locally
13. Security
13.1. Authentication and Cloud Foundry
14. Application Names and Prefixes
14.1. Using Custom Routes
15. Deploying Docker Applications
16. Configuration Reference
16.1. Understanding what’s going on
16.2. Using Spring Cloud Config Server
16.2.1. Stream, Task, and Spring Cloud Config Server
16.2.2. Sample Manifest Template
16.2.3. Self-signed SSL Certificate and Spring Cloud Config Server
17. Application Level Service Bindings
18. A Note About User Provided Services
19. Application Rolling Upgrades
20. Maximum Disk Quota Configuration
20.1. PCF’s Operations Manager Configuration
20.2. Scale Application
20.3. Configuring target free disk percentage
IV. Server Configuration
21. Feature Toggles
22. Database Configuration
23. Security
23.1. Enabling HTTPS
23.1.1. Using Self-Signed Certificates
23.1.2. Self-Signed Certificates and the Shell
23.2. Basic Authentication
23.2.1. File based authentication
23.2.2. LDAP Authentication
LDAP Transport Security
23.2.3. Customizing authorization
23.2.4. Authorization - Shell and Dashboard Behavior
23.2.5. Authorization with Ldap
23.3. OAuth 2.0
23.3.1. Authentication using the Spring Cloud Data Flow Shell
23.3.2. OAuth2 Authentication Examples
Local OAuth2 Server
Authentication using GitHub
23.4. Securing the Spring Boot Management Endpoints
24. Monitoring and Management
24.1. Spring Boot Admin
24.2. Monitoring Deployed Applications
V. Streams
25. Introduction
26. Stream DSL
27. Register a Stream App
27.1. Whitelisting application properties
27.2. Creating and using a dedicated metadata artifact
27.2.1. Using the companion artifact
28. Creating custom applications
29. Creating a Stream
29.1. Application properties
29.1.1. Passing application properties when creating a stream
29.2. Deployment properties
29.2.1. Application properties versus Deployer properties
29.2.2. Passing instance count as deployment property
29.2.3. Inline vs file reference properties
29.2.4. Passing application properties when deploying a stream
29.2.5. Passing Spring Cloud Stream properties for the application
29.2.6. Passing per-binding producer consumer properties
29.2.7. Passing stream partition properties during stream deployment
29.2.8. Passing application content type properties
29.2.9. Overriding application properties during stream deployment
30. Destroying a Stream
31. Deploying and Undeploying Streams
32. Other Source and Sink Application Types
33. Simple Stream Processing
34. Stateful Stream Processing
35. Tap a Stream
36. Using Labels in a Stream
37. Explicit Broker Destinations in a Stream
38. Directed Graphs in a Stream
38.1. Common application properties
39. Stream applications with multiple binder configurations
VI. Tasks
40. Introducing Spring Cloud Task
41. The Lifecycle of a task
41.1. Creating a custom Task Application
41.2. Registering a Task Application
41.3. Creating a Task
41.4. Launching a Task
41.5. Reviewing Task Executions
41.6. Destroying a Task
42. Task Repository
42.1. Configuring the Task Execution Repository
42.1.1. Local
42.1.2. Task Application Repository
42.2. Datasource
43. Subscribing to Task/Batch Events
44. Launching Tasks from a Stream
44.1. TriggerTask
44.2. Translator
VII. Tasks on Cloud Foundry
45. Version Compatibility
46. Tooling
47. Task Database Schema
48. Running Task Applications
48.1. Create a Task
48.2. Launch a Task
48.3. View Task Logs
48.4. List Tasks
48.5. List Task Executions
48.6. Destroy a Task
48.7. Deleting Task From Cloud Foundry
VIII. Dashboard
49. Introduction
50. Apps
50.1. Bulk Import of Applications
51. Runtime
52. Streams
53. Create Stream
54. Tasks
54.1. Apps
54.1.1. Create a Task Definition from a selected Task App
54.1.2. View Task App Details
54.2. Definitions
54.2.1. Creating Task Definitions using the bulk define interface
54.2.2. Launching Tasks
54.3. Executions
55. Jobs
55.1. List job executions
55.1.1. Job execution details
55.1.2. Step execution details
55.1.3. Step Execution Progress
56. Analytics
IX. ‘How-to’ guides
57. Configure Maven Properties
58. Logging
58.1. Deployment Logs
58.2. Application Logs
X. Appendices
A. Migrating from Spring XD to Spring Cloud Data Flow
A.1. Terminology Changes
A.2. Modules to Applications
A.2.1. Custom Applications
A.2.2. Application Registration
A.2.3. Application Properties
A.3. Message Bus to Binders
A.3.1. Message Bus
A.3.2. Binders
A.3.3. Named Channels
A.3.4. Directed Graphs
A.4. Batch to Tasks
A.5. Shell/DSL Commands
A.6. REST-API
A.7. UI / Flo
A.8. Architecture Components
A.8.1. ZooKeeper
A.8.2. RDBMS
A.8.3. Redis
A.8.4. Cluster Topology
A.9. Central Configuration
A.10. Distribution
A.11. Hadoop Distribution Compatibility
A.12. YARN Deployment
A.13. Use Case Comparison
A.13.1. Use Case #1
A.13.2. Use Case #2
A.13.3. Use Case #3
B. Building
B.1. Basic Compile and Test
B.2. Documentation
B.3. Working with the code
B.3.1. Importing into eclipse with m2eclipse
B.3.2. Importing into eclipse without m2eclipse
   Next
   Part I. Spring Cloud Data Flow for Cloud Foundry