Spring Cloud Data Flow Samples

Authors

Sabby Anandan, David Turanski, Glenn Renfro, Eric Bottard, Mark Pollack, Chris Schaefer, Christian Tzolov, Gunnar Hillert

1.0.0.BUILD-SNAPSHOT

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.


Table of Contents

1. Overview
2. Java DSL
2.1. Deploying a stream programmaticaly
2.1.1. Step 1 Build the sample application
2.1.2. Step 2 Start Data Flow and run the sample application
2.1.3. Step 3 Post some data to the server
2.1.4. Step 4 Verify the output
2.1.5. Step 5 Use Authentication
3. Streaming
3.1. HTTP to Cassandra Demo
3.1.1. Prerequisites
3.1.2. Running Locally
Additional Prerequisites
Building and Running the Demo
3.1.3. Running on Cloud Foundry
Additional Prerequisites
Running the Demo
3.1.4. Summary
3.2. HTTP to MySQL Demo
3.2.1. Prerequisites
3.2.2. Running Locally
Additional Prerequisites
Building and Running the Demo
3.2.3. Running on Cloud Foundry
Additional Prerequisites
Building and Running the Demo
3.2.4. Summary
3.3. HTTP to Gemfire Demo
3.3.1. Prerequisites
3.3.2. Running Locally
Additional Prerequisites
Building and Running the Demo
3.3.3. Running on Cloud Foundry
Additional Prerequisites
Building and Running the Demo
3.3.4. Summary
3.4. Gemfire CQ to Log Demo
3.4.1. Prerequisites
3.4.2. Running Locally
Additional Prerequisites
Building and Running the Demo
3.4.3. Running on Cloud Foundry
Additional Prerequisites
Building and Running the Demo
3.4.4. Summary
3.5. Gemfire to Log Demo
3.5.1. Prerequisites
3.5.2. Running Locally
Additional Prerequisites
Building and Running the Demo
3.5.3. Running on Cloud Foundry
Additional Prerequisites
Building and Running the Demo
3.5.4. Summary
3.6. Custom Spring Cloud Stream Processor
3.6.1. Prerequisites
3.6.2. Creating the Custom Stream App
3.6.3. Deploying the App to Spring Cloud Data Flow
3.6.4. Summary
4. Task / Batch
4.1. Batch Job on Cloud Foundry
4.1.1. Prerequisites
4.1.2. Building and Running the Demo
4.1.3. Summary
4.2. Batch File Ingest
4.2.1. Prerequisites
4.2.2. Batch File Ingest Demo Overview
4.2.3. Building and Running the Demo
4.2.4. Summary
5. Stream Launching Batch Job
5.1. Batch File Ingest - SFTP Demo
5.1.1. Prerequisites
5.1.2. Running Locally
Additional Prerequisites
Building and Running the Demo
5.1.3. Running on Cloud Foundry
Additional Prerequisites
Configuring the SCDF and Skipper servers
Running the Demo
5.1.4. Running on Kubernetes
Additional Prerequisites
Running the Demo
5.1.5. Limiting Concurrent Task Executions
Configuring the SCDF server
Running the demo
5.1.6. Avoid Duplicate Processing
5.1.7. Summary
6. Analytics
6.1. Twitter Analytics
6.1.1. Prerequisites
6.1.2. Building and Running the Demo
6.1.3. Summary
7. Data Science
7.1. Species Prediction
7.1.1. Prerequisites
7.1.2. Building and Running the Demo
7.1.3. Summary
8. Functions
8.1. Functions in Spring Cloud Data Flow
8.1.1. Prerequisites
8.1.2. Building and Running the Demo
8.1.3. Summary
9. Monitoring
9.1. Spring Cloud Data Flow Monitoring
10. Security
10.1. LDAP Security and UAA Example
10.1.1. Requirements
10.1.2. Build + Start LDAP Server
10.1.3. Download + Start UAA Server
10.1.4. Prepare UAA Server
10.1.5. Quick Test Using Curl
10.1.6. Download + Start Spring Cloud Skipper
10.1.7. Download + Start Spring Cloud Data Flow
10.1.8. Helper Utility
10.1.9. Configure and run a Composed Task