Class ConsumerProperties

  • Direct Known Subclasses:
    ContainerProperties

    public class ConsumerProperties
    extends java.lang.Object
    Common consumer properties.
    Since:
    2.3
    Author:
    Gary Russell
    • Constructor Summary

      Constructors 
      Constructor Description
      ConsumerProperties​(java.lang.String... topics)
      Create properties for a container that will subscribe to the specified topics.
      ConsumerProperties​(java.util.regex.Pattern topicPattern)
      Create properties for a container that will subscribe to topics matching the specified pattern.
      ConsumerProperties​(TopicPartitionOffset... topicPartitions)
      Create properties for a container that will assign itself the provided topic partitions.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.time.Duration getAuthorizationExceptionRetryInterval()  
      java.lang.String getClientId()
      Return the client id.
      org.apache.kafka.clients.consumer.OffsetCommitCallback getCommitCallback()
      Return the commit callback.
      LogIfLevelEnabled.Level getCommitLogLevel()
      The level at which to log offset commits.
      int getCommitRetries()
      The number of retries allowed when a RetriableCommitFailedException is thrown by the consumer.
      org.apache.kafka.clients.consumer.ConsumerRebalanceListener getConsumerRebalanceListener()
      Return the rebalance listener.
      java.lang.String getGroupId()
      Return the container's group id.
      java.util.Properties getKafkaConsumerProperties()
      Get the consumer properties that will be merged with the consumer properties provided by the consumer factory; properties here will supersede any with the same name(s) in the consumer factory.
      long getPollTimeout()  
      java.time.Duration getSyncCommitTimeout()
      Return the sync commit timeout.
      TopicPartitionOffset[] getTopicPartitions()
      Return the configured TopicPartitionOffsets.
      java.util.regex.Pattern getTopicPattern()
      Return the configured topic pattern.
      java.lang.String[] getTopics()
      Return the configured topics.
      boolean isFixTxOffsets()
      Whether or not to correct terminal transactional offsets.
      boolean isOnlyLogRecordMetadata()  
      boolean isSyncCommits()  
      protected java.lang.String renderProperties()  
      void setAuthorizationExceptionRetryInterval​(java.time.Duration authorizationExceptionRetryInterval)
      Set the interval between retries after AuthorizationException is thrown by KafkaConsumer.
      void setClientId​(java.lang.String clientId)
      Set the client id; overrides the consumer factory client.id property.
      void setCommitCallback​(org.apache.kafka.clients.consumer.OffsetCommitCallback commitCallback)
      Set the commit callback; by default a simple logging callback is used to log success at DEBUG level and failures at ERROR level.
      void setCommitLogLevel​(LogIfLevelEnabled.Level commitLogLevel)
      Set the level at which to log offset commits.
      void setCommitRetries​(int commitRetries)
      Set number of retries allowed when a RetriableCommitFailedException is thrown by the consumer.
      void setConsumerRebalanceListener​(org.apache.kafka.clients.consumer.ConsumerRebalanceListener consumerRebalanceListener)
      Set the user defined ConsumerRebalanceListener implementation.
      void setFixTxOffsets​(boolean fixTxOffsets)
      When consuming records produced by a transactional producer, and the consumer is positioned at the end of a partition, the lag can incorrectly be reported as greater than zero, due to the pseudo record used to indicate transaction commit/rollback and, possibly, the presence of rolled-back records.
      void setGroupId​(java.lang.String groupId)
      Set the group id for this container.
      void setKafkaConsumerProperties​(java.util.Properties kafkaConsumerProperties)
      Set the consumer properties that will be merged with the consumer properties provided by the consumer factory; properties here will supersede any with the same name(s) in the consumer factory.
      void setOnlyLogRecordMetadata​(boolean onlyLogRecordMetadata)
      Set to false to log record.toString() in log messages instead of [email protected].
      void setPollTimeout​(long pollTimeout)
      Set the max time to block in the consumer waiting for records.
      void setSyncCommits​(boolean syncCommits)
      Set whether or not to call consumer.commitSync() or commitAsync() when the container is responsible for commits.
      void setSyncCommitTimeout​(java.time.Duration syncCommitTimeout)
      Set the timeout for commitSync operations (if isSyncCommits().
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • ConsumerProperties

        public ConsumerProperties​(java.lang.String... topics)
        Create properties for a container that will subscribe to the specified topics.
        Parameters:
        topics - the topics.
      • ConsumerProperties

        public ConsumerProperties​(java.util.regex.Pattern topicPattern)
        Create properties for a container that will subscribe to topics matching the specified pattern. The framework will create a container that subscribes to all topics matching the specified pattern to get dynamically assigned partitions. The pattern matching will be performed periodically against topics existing at the time of check.
        Parameters:
        topicPattern - the pattern.
        See Also:
        CommonClientConfigs.METADATA_MAX_AGE_CONFIG
      • ConsumerProperties

        public ConsumerProperties​(TopicPartitionOffset... topicPartitions)
        Create properties for a container that will assign itself the provided topic partitions.
        Parameters:
        topicPartitions - the topic partitions.
    • Method Detail

      • getTopics

        @Nullable
        public java.lang.String[] getTopics()
        Return the configured topics.
        Returns:
        the topics.
      • getTopicPattern

        @Nullable
        public java.util.regex.Pattern getTopicPattern()
        Return the configured topic pattern.
        Returns:
        the topic pattern.
      • setPollTimeout

        public void setPollTimeout​(long pollTimeout)
        Set the max time to block in the consumer waiting for records.
        Parameters:
        pollTimeout - the timeout in ms; default 5000L.
      • getPollTimeout

        public long getPollTimeout()
      • setGroupId

        public void setGroupId​(java.lang.String groupId)
        Set the group id for this container. Overrides any group.id property provided by the consumer factory configuration.
        Parameters:
        groupId - the group id.
      • getGroupId

        @Nullable
        public java.lang.String getGroupId()
        Return the container's group id.
        Returns:
        the group id.
      • getClientId

        public java.lang.String getClientId()
        Return the client id.
        Returns:
        the client id.
        See Also:
        setClientId(String)
      • setClientId

        public void setClientId​(java.lang.String clientId)
        Set the client id; overrides the consumer factory client.id property. When used in a concurrent container, will be suffixed with '-n' to provide a unique value for each consumer.
        Parameters:
        clientId - the client id.
      • setConsumerRebalanceListener

        public void setConsumerRebalanceListener​(org.apache.kafka.clients.consumer.ConsumerRebalanceListener consumerRebalanceListener)
        Set the user defined ConsumerRebalanceListener implementation.
        Parameters:
        consumerRebalanceListener - the ConsumerRebalanceListener instance
      • getConsumerRebalanceListener

        @Nullable
        public org.apache.kafka.clients.consumer.ConsumerRebalanceListener getConsumerRebalanceListener()
        Return the rebalance listener.
        Returns:
        the listener.
      • setSyncCommitTimeout

        public void setSyncCommitTimeout​(@Nullable
                                         java.time.Duration syncCommitTimeout)
        Set the timeout for commitSync operations (if isSyncCommits(). Overrides the default api timeout property.
        Parameters:
        syncCommitTimeout - the timeout.
        See Also:
        setSyncCommits(boolean)
      • getSyncCommitTimeout

        @Nullable
        public java.time.Duration getSyncCommitTimeout()
        Return the sync commit timeout.
        Returns:
        the timeout.
      • setCommitCallback

        public void setCommitCallback​(org.apache.kafka.clients.consumer.OffsetCommitCallback commitCallback)
        Set the commit callback; by default a simple logging callback is used to log success at DEBUG level and failures at ERROR level. Used when syncCommits is false.
        Parameters:
        commitCallback - the callback.
        See Also:
        setSyncCommits(boolean)
      • getCommitCallback

        @Nullable
        public org.apache.kafka.clients.consumer.OffsetCommitCallback getCommitCallback()
        Return the commit callback.
        Returns:
        the callback.
      • isSyncCommits

        public boolean isSyncCommits()
      • getCommitLogLevel

        public LogIfLevelEnabled.Level getCommitLogLevel()
        The level at which to log offset commits.
        Returns:
        the level.
      • setCommitLogLevel

        public void setCommitLogLevel​(LogIfLevelEnabled.Level commitLogLevel)
        Set the level at which to log offset commits. Default: DEBUG.
        Parameters:
        commitLogLevel - the level.
      • getKafkaConsumerProperties

        public java.util.Properties getKafkaConsumerProperties()
        Get the consumer properties that will be merged with the consumer properties provided by the consumer factory; properties here will supersede any with the same name(s) in the consumer factory. You can add non-String-valued properties, but the property name (hashtable key) must be String; all others will be ignored. group.id and client.id are ignored.
        Returns:
        the properties.
        See Also:
        ConsumerConfig, setGroupId(String), setClientId(String)
      • setKafkaConsumerProperties

        public void setKafkaConsumerProperties​(java.util.Properties kafkaConsumerProperties)
        Set the consumer properties that will be merged with the consumer properties provided by the consumer factory; properties here will supersede any with the same name(s) in the consumer factory. group.id and client.id are ignored. Property keys must be Strings.
        Parameters:
        kafkaConsumerProperties - the properties.
        See Also:
        ConsumerConfig, setGroupId(String), setClientId(String)
      • getAuthorizationExceptionRetryInterval

        public java.time.Duration getAuthorizationExceptionRetryInterval()
      • setAuthorizationExceptionRetryInterval

        public void setAuthorizationExceptionRetryInterval​(java.time.Duration authorizationExceptionRetryInterval)
        Set the interval between retries after AuthorizationException is thrown by KafkaConsumer. By default the field is null and retries are disabled. In such case the container will be stopped. The interval must be less than max.poll.interval.ms consumer property.
        Parameters:
        authorizationExceptionRetryInterval - the duration between retries
        Since:
        2.3.5
      • getCommitRetries

        public int getCommitRetries()
        The number of retries allowed when a RetriableCommitFailedException is thrown by the consumer.
        Returns:
        the number of retries.
        Since:
        2.3.9
      • setCommitRetries

        public void setCommitRetries​(int commitRetries)
        Set number of retries allowed when a RetriableCommitFailedException is thrown by the consumer. Default 3 (4 attempts total).
        Parameters:
        commitRetries - the commitRetries.
        Since:
        2.3.9
      • isOnlyLogRecordMetadata

        public boolean isOnlyLogRecordMetadata()
      • setOnlyLogRecordMetadata

        public void setOnlyLogRecordMetadata​(boolean onlyLogRecordMetadata)
        Set to false to log record.toString() in log messages instead of [email protected].
        Parameters:
        onlyLogRecordMetadata - false to log the entire record.
        Since:
        2.2.14
      • isFixTxOffsets

        public boolean isFixTxOffsets()
        Whether or not to correct terminal transactional offsets.
        Returns:
        true to fix.
        Since:
        2.5.6
        See Also:
        setFixTxOffsets(boolean)
      • setFixTxOffsets

        public void setFixTxOffsets​(boolean fixTxOffsets)
        When consuming records produced by a transactional producer, and the consumer is positioned at the end of a partition, the lag can incorrectly be reported as greater than zero, due to the pseudo record used to indicate transaction commit/rollback and, possibly, the presence of rolled-back records. This does not functionally affect the consumer but some users have expressed concern that the "lag" is non-zero. Set this to true and the container will correct such mis-reported offsets. The check is performed before the next poll to avoid adding significant complexity to the commit processing. IMPORTANT: At the time of writing, the lag will only be corrected if the consumer is configured with isolation.level=read_committed and max.poll.records is greater than 1. See https://issues.apache.org/jira/browse/KAFKA-10683 for more information.
        Parameters:
        fixTxOffsets - true to correct the offset(s).
        Since:
        2.5.6
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • renderProperties

        protected final java.lang.String renderProperties()