What’s new?

This version requires 3.7.0 kafka-clients. The 3.7.0 version of Kafka client introduces the new consumer group protocol. Fore more details and it’s limitations see KIP-848. The new consumer group protocol is an early access release and not meant to be used in production. It is only recommended to use for testing purposes in this version. Therefore, Spring for Apache Kafka supports this new consumer group protocol only to the extent of such testing level support available in the kafka-client itself. By default, Spring for Apache Kafka uses the classic consumer group protocol and when testing the new consumer group protocol, that needs to be opted-in via the group.protocol property on the consumer.

The kraft mode is disabled in EmbeddedKafka by default and users wanting to use the kraft mode must enable it. This is due to certain instabilities observed while using EmbeddedKafka in kraft mode, especially when testing the new consumer group protocol. The new consumer group protocol is only supported in kraft mode and because of this, when testing the new protocol, that needs to be done against a real Kafka cluster and not the one based on the KafkaClusterTestKit, which EmbeddedKafka is based upon. In addition, there were some other race conditions observed, while running multiple KafkaListener methods with EmbeddedKafka in kraft mode. Until these issues are resolved, the kraft default on EmbeddedKafka will remain as false.

A new API KafkaStreamsInteractiveQuerySupport for accessing queryable stores used in Kafka Streams interactive queries. See Kafka Streams Interactive Support for more details.

A new TransactionIdSuffixStrategy interface was introduced to manage transactional.id suffix. The default implementation is DefaultTransactionIdSuffixStrategy when setting maxCache greater than zero can reuse transactional.id within a specific range, otherwise suffixes will be generated on the fly by incrementing a counter. See Fixed TransactionIdSuffix for more information.

@KafkaListener (and @KafkaHandler) methods can now return asynchronous return types include CompletableFuture<?>, Mono<?> and Kotlin suspend functions. See Async Returns for more information.

It’s now possible to redirect messages to the custom DLTs based on the type of the exception, which has been thrown during the message processing. Rules for the redirection are set either via the RetryableTopic.exceptionBasedDltRouting or the RetryTopicConfigurationBuilder.dltRoutingRules. Custom DLTs are created automatically as well as other retry and dead-letter topics. See Routing of messages to custom DLTs based on thrown exceptions for more information.

Deprecating the transactionManager property in ContainerProperties in favor of KafkaAwareTransactionManager, a narrower type compared to the general PlatformTransactionManager. See ContainerProperties and Transaction Synchronization.

A new AfterRollbackProcessor API processBatch is provided. See After-rollback Processor for more information.

Change @RetryableTopic property SameIntervalTopicReuseStrategy default value to SINGLE_TOPIC. See Single Topic for maxInterval Exponential Delay.

Non-blocking retries support @KafkaListener on a Class. See Non-Blocking Retries.

Provides a new public API to find RetryTopicConfiguration. See Find RetryTopicConfiguration

The RetryTopicConfigurer support process and register MultiMethodKafkaListenerEndpoint. The MultiMethodKafkaListenerEndpoint provides getter/setter for properties defaultMethod and methods. Modify the EndpointCustomizer that strictly for MethodKafkaListenerEndpoint types. The EndpointHandlerMethod add new constructors construct an instance for the provided bean. Provides new class EndpointHandlerMultiMethod to handler multi method for retrying endpoints.

ConsumerCallback provides a new API to seek to an offset based on a user-defined function, which takes the current offset in the consumer as an argument. See Seek API Docs for more details.

Adding seekPosition property to @PartitionOffset support for TopicPartitionOffset.SeekPosition. See manual-assignment for more details.

TopicPartitionOffset has a new constructor that takes a user-provided function to compute the offset to seek to. When this constructor is used, the framework calls the function with the input argument of the current consumer offset position. See Seek API Docs for more details.

For Spring Boot applications which define an application name, this name is now used as a default prefix for auto-generated client IDs for certain client types. See Default client ID prefixes for more details.