Spring XD Guide

Authors

Mark Fisher, Mark Pollack, David Turanski, Gunnar Hillert, Eric Bottard, Gary Russell, Ilayaperumal Gopinathan, Jennifer Hickey, Michael Minella, Luke Taylor, Thomas Risberg, Winston Koh

1.0.0


Table of Contents

I. Reference Guide
1. Introduction
1.1. Overview
2. Getting Started
2.1. Requirements
2.2. Download Spring XD
2.3. Install Spring XD
2.4. Start the Runtime
2.5. Create a Stream
2.6. Explore Spring XD
3. Running in Distributed Mode
3.1. Introduction
3.2. Installing Redis
3.2.1. Redis on Windows
3.2.2. Redis is not running
3.3. Starting Redis
3.4. Starting Spring XD in Distributed Mode
4. Architecture
4.1. Introduction
4.1.1. Runtime Architecture
DIRT Runtime
Support for other distributed runtimes
4.1.2. Single Node Runtime
4.1.3. Admin Server Architecture
4.1.4. Container Server Architecture
4.1.5. Streams
4.1.6. Stream Deployment
4.2. Jobs
4.3. Taps
5. Streams
5.1. Introduction
5.2. Creating a Simple Stream
5.3. Deleting a Stream
5.4. Other Source and Sink Types
5.5. Simple Stream Processing
5.6. DSL Syntax
6. Modules
6.1. Introduction
6.2. Creating a Module
6.2.1. Modules and Spring
6.2.2. Integration Modules
6.3. Registering a Module
7. Sources
7.1. Introduction
7.1.1. HTTP
HTTP with options
7.1.2. Tail
Tail with options
Tail Status Events
7.1.3. Twitter Search
7.1.4. GemFire Continuous Query (CQ)
Launching the XD GemFire Server
Options
7.1.5. Syslog
7.1.6. TCP
TCP with options
Available Decoders
Examples
Binary Data Example
8. Processors
8.1. Introduction
8.1.1. Filter
Filter with SpEL expression
Filter with Groovy Script
8.1.2. JSON Field Value Filter
8.1.3. Transform
Transform with SpEL expression
Transform with Groovy Script
8.1.4. JSON Field Extractor
8.1.5. Script
9. Sinks
9.1. Introduction
9.1.1. File Sink
9.1.2. Hadoop (HDFS)
HDFS with Options
9.1.3. TCP
TCP with Options
Available Encoders
An Additional Example
9.1.4. GemFire Server
Launching the XD GemFire Server
Gemfire sinks
Example
10. Taps
10.1. Introduction
10.2. Tap Lifecycle
11. Analytics
11.1. Introdution
11.2. Counter
11.3. Field Value Counter
11.4. Gauge
11.4.1. Simple Tap Example
11.5. Rich Gauge
11.5.1. Simple Tap Example
11.5.2. Stock Price Example
11.5.3. Improved Stock Price Example
12. DSL Reference
12.1. Introduction
12.2. Pipes and filters
12.3. Module parameters
12.4. Tap
13. Tuples
13.1. Introduction
13.1.1. Creating a Tuple
13.1.2. Getting Tuple values
13.1.3. Using SpEL expressions to filter a tuple
14. Samples
14.1. Syslog ingestion into HDFS
14.1.1. A sample configuration using syslog-ng
II. Appendices
A. Installing Hadoop
A.1. Installing Hadoop
A.2. Running Hadoop
B. Creating a Source Module
B.1. Introduction
B.2. Create the module Application Context file
B.2.1. Make the module configurable
B.3. Test the module locally
B.3.1. Create a project
B.3.2. Create the Spring integration test
B.4. Deploy the module
B.5. Test the deployed module
C. Creating a Processor Module
C.1. Introduction
C.2. Write the Transformer Code
C.3. Create the module Application Context File
C.4. Deploy the Module
C.5. Test the deployed module
D. Creating a Sink Module
D.1. Introduction
D.2. Create the module Application Context file
D.3. Make the module configurable
D.4. Test the module locally
D.4.1. Create a project
D.4.2. Create the Spring integration test
D.4.3. Run the test
D.5. Deploy the module
D.6. Test the deployed module