Metadata Store

Many external systems, services, or resources are not transactional (Twitter, RSS, file systems, and so on), and there is not any ability to mark the data as read. Also, sometimes, you may need to implement the Enterprise Integration Pattern idempotent receiver in some integration solutions. To achieve this goal and store some previous state of the endpoint before the next interaction with external system or to deal with the next message, Spring Integration provides the metadata store component as an an implementation of the org.springframework.integration.metadata.MetadataStore interface with a general key-value contract.

The metadata store is designed to store various types of generic metadata (for example, the published date of the last feed entry that has been processed) to help components such as the feed adapter deal with duplicates. If a component is not directly provided with a reference to a MetadataStore, the algorithm for locating a metadata store is as follows: First, look for a bean with a metadataStore ID in the application context. If one is found, use it. Otherwise, create a new instance of SimpleMetadataStore, which is an in-memory implementation that persists only metadata within the lifecycle of the currently running application context. This means that, upon restart, you may end up with duplicate entries.

If you need to persist metadata between application context restarts, the framework provides the following persistent MetadataStores:

The PropertiesPersistingMetadataStore is backed by a properties file and a PropertiesPersister.

By default, it persists only the state when the application context is closed normally. It implements Flushable so that you can persist the state at will, by invoking flush(). The following example shows how to configure a 'PropertiesPersistingMetadataStore' with XML:

<bean id="metadataStore"
    class="org.springframework.integration.metadata.PropertiesPersistingMetadataStore"/>

Alternatively, you can provide your own implementation of the MetadataStore interface (for example, JdbcMetadataStore) and configure it as a bean in the application context.

Starting with version 4.0, SimpleMetadataStore, PropertiesPersistingMetadataStore, and RedisMetadataStore implement ConcurrentMetadataStore. These provide for atomic updates and can be used across multiple component or application instances.

Idempotent Receiver and Metadata Store

The metadata store is useful for implementing the EIP idempotent receiver pattern when there is need to filter an incoming message if it has already been processed and you can discard it or perform some other logic on discarding. The following configuration shows an example of how to do so:

<int:filter input-channel="serviceChannel"
			output-channel="idempotentServiceChannel"
			discard-channel="discardChannel"
			expression="@metadataStore.get(headers.businessKey) == null"/>

<int:publish-subscribe-channel id="idempotentServiceChannel"/>

<int:outbound-channel-adapter channel="idempotentServiceChannel"
                              expression="@metadataStore.put(headers.businessKey, '')"/>

<int:service-activator input-channel="idempotentServiceChannel" ref="service"/>

The value of the idempotent entry may be an expiration date, after which that entry should be removed from metadata store by some scheduled reaper.

See also Idempotent Receiver Enterprise Integration Pattern.

MetadataStoreListener

Some metadata stores (currently only zookeeper) support registering a listener to receive events when items change, as the following example shows:

public interface MetadataStoreListener {

	void onAdd(String key, String value);

	void onRemove(String key, String oldValue);

	void onUpdate(String key, String newValue);
}

See the Javadoc for more information. The MetadataStoreListenerAdapter can be subclassed if you are interested only in a subset of events.