Spring for Apache Kafka

Authors

Gary Russell , Artem Bilan , Biju Kunjummen

2.2.2.RELEASE

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.


Table of Contents

1. Preface
2. What’s new?
2.1. What’s new in 2.2 Since 2.1
2.1.1. Kafka Client Version
2.1.2. Class/Package Changes
2.1.3. After rollback processing
2.1.4. ConcurrentKafkaListenerContainerFactory changes
2.1.5. Listener Container Changes
2.1.6. @KafkaListener Changes
2.1.7. Header Mapping Changes
2.1.8. Embedded Kafka Changes
2.1.9. JsonSerializer/Deserializer Enhancements
2.1.10. Kafka Streams Changes
2.1.11. Transactional Id
3. Introduction
3.1. Quick Tour for the Impatient
3.1.1. Introduction
Compatibility
Very, Very Quick
With Java Configuration
Even Quicker, with Spring Boot
4. Reference
4.1. Using Spring for Apache Kafka
4.1.1. Configuring Topics
4.1.2. Sending Messages
KafkaTemplate
Transactions
ReplyingKafkaTemplate
4.1.3. Receiving Messages
Message Listeners
Message Listener Containers
@KafkaListener Annotation
Container Thread Naming
@KafkaListener as a Meta Annotation
@KafkaListener on a Class
@KafkaListener Lifecycle Management
@KafkaListener @Payload Validation
Rebalance Listeners
Forwarding Listener Results using @SendTo
Filtering Messages
Retrying Deliveries
Stateful Retry
Detecting Idle and Non-Responsive Consumers
Topic/Partition Initial Offset
Seeking to a Specific Offset
Container factory
Thread Safety
4.1.4. Pausing/Resuming Listener Containers
4.1.5. Events
4.1.6. Serialization/Deserialization and Message Conversion
Overview
Spring Messaging Message Conversion
ErrorHandlingDeserializer
Payload Conversion with Batch Listeners
ConversionService Customization
4.1.7. Message Headers
4.1.8. Null Payloads and Log Compaction Tombstone Records
4.1.9. Handling Exceptions
Listener Error Handlers
Container Error Handlers
Consumer-Aware Container Error Handlers
Seek To Current Container Error Handlers
Container Stopping Error Handlers
After Rollback Processor
Publishing Dead-Letter Records
4.1.10. Kerberos
4.2. Kafka Streams Support
4.2.1. Introduction
4.2.2. Basics
4.2.3. Spring Management
4.2.4. JSON Serdes
4.2.5. Configuration
4.2.6. Kafka Streams Example
4.3. Testing Applications
4.3.1. Introduction
4.3.2. JUnit
4.3.3. Configuring Topics
4.3.4. Using the Same Broker(s) for Multiple Test Classes
4.3.5. @EmbeddedKafka Annotation
4.3.6. Embedded Broker in @SpringBootTest s
JUnit4 Class Rule
@EmbeddedKafka Annotation or EmbeddedKafkaBroker Bean
4.3.7. Hamcrest Matchers
4.3.8. AssertJ Conditions
4.3.9. Example
5. Spring Integration
5.1. Spring Integration for Apache Kafka
5.1.1. Introduction
5.1.2. Outbound Channel Adapter
5.1.3. Message Driven Channel Adapter
5.1.4. Outbound Gateway
5.1.5. Inbound Gateway
5.1.6. Message Conversion
5.1.7. Null Payloads and Log Compaction Tombstone Records
5.1.8. What’s New in Spring Integration for Apache Kafka
2.1.x
2.2.x
2.3.x
3.0.x
3.1.x
6. Other Resources
A. Override Dependencies to use the 2.1.x kafka-clients with an Embedded Broker
B. Change History
B.1. Changes between 2.0 and 2.1
B.1.1. Kafka Client Version
B.1.2. JSON Improvements
B.1.3. Container Stopping Error Handlers
B.1.4. Pausing/Resuming Containers
B.1.5. Stateful Retry
B.1.6. Client ID
B.1.7. Logging Offset Commits
B.1.8. Default @KafkaHandler
B.1.9. ReplyingKafkaTemplate
B.1.10. ChainedKafkaTransactionManager
B.1.11. Migration Guide from 2.0
B.2. Changes Between 1.3 and 2.0
B.2.1. Spring Framework and Java Versions
B.2.2. @KafkaListener Changes
B.2.3. Message Listeners
B.2.4. ConsumerAwareRebalanceListener
B.3. Changes Between 1.2 and 1.3
B.3.1. Support for Transactions
B.3.2. Support for Headers
B.3.3. Creating Topics
B.3.4. Support for Kafka timestamps
B.3.5. @KafkaListener Changes
B.3.6. @EmbeddedKafka Annotation
B.3.7. Kerberos Configuration
B.4. Changes between 1.1 and 1.2
B.5. Changes between 1.0 and 1.1
B.5.1. Kafka Client
B.5.2. Batch Listeners
B.5.3. Null Payloads
B.5.4. Initial Offset
B.5.5. Seek