Similar to Spring XD, there’s an abstraction available to extend the binder interface. By default, we take the opinionated view of Apache Kafka and RabbitMQ as the production-ready binders and are available as GA releases. We also have an experimental version of the Gemfire binder.

Selecting a binder is as simple as providing the right binder dependency in the classpath. If you’re to choose Kafka as the binder, you’d register stream applications that are pre-built with Kafka binder in it. If you were to create a custom application with Kafka binder, you’d add the following dependency in the classpath.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
    <version>1.0.2.RELEASE</version>
</dependency>

Fundamentally, all the messaging channels are backed by pub/sub semantics. Unlike Spring XD, the messaging channels are backed only by topics or topic-exchange and there’s no representation of queues in the new architecture.

If you’re building non-linear streams, you could take advantage of named destinations to build directed graphs.

for instance, in Spring XD:

stream create f --definition "queue:foo > transform --expression=payload+'-foo' | log" --deploy
stream create b --definition "queue:bar > transform --expression=payload+'-bar' | log" --deploy
stream create r --definition "http | router --expression=payload.contains('a')?'queue:foo':'queue:bar'" --deploy

for instance, in Spring Cloud Data Flow:

stream create f --definition ":foo > transform --expression=payload+'-foo' | log" --deploy
stream create b --definition ":bar > transform --expression=payload+'-bar' | log" --deploy
stream create r --definition "http | router --expression=payload.contains('a')?':foo':':bar'" --deploy

ZooKeeper is not used in the new architecture.

Spring Cloud Data Flow uses an RDBMS instead of Redis for stream/task definitions, application registration, and for job repositories.The default configuration uses an embedded H2 instance, but Oracle, DB2, SqlServer, MySQL/MariaDB, PostgreSQL, H2, and HSQLDB databases are supported. To use Oracle, DB2 and SqlServer you will need to create your own Data Flow Server using Spring Initializr and add the appropriate JDBC driver dependency.

Running a Redis cluster is only required for analytics functionality. Specifically, when the counter-sink, field-value-counter-sink, or aggregate-counter-sink applications are used, it is expected to also have a running instance of Redis cluster.

Spring XD’s xd-admin and xd-container server components are replaced by stream and task applications themselves running as autonomous Spring Boot applications. The applications run natively on various platforms including Cloud Foundry, Apache YARN, Apache Mesos, or Kubernetes. You can develop, test, deploy, scale +/-, and interact with (Spring Boot) applications individually, and they can evolve in isolation.

(It is assumed both XD and SCDF distributions are already downloaded)

Description: Simple ticktock example using local/singlenode.

(It is assumed both XD and SCDF distributions are already downloaded)

Description: Stream with custom module/application.

(It is assumed both XD and SCDF distributions are already downloaded)

Description: Simple batch-job.