Mail Support

This section describes how to work with mail messages in Spring Integration.

You need to include this dependency into your project:

Maven

<dependency>
    <groupId>org.springframework.integration</groupId>
    <artifactId>spring-integration-mail</artifactId>
    <version>5.3.0.RELEASE</version>
</dependency>

Gradle

compile "org.springframework.integration:spring-integration-mail:5.3.0.RELEASE"

The javax.mail:javax.mail-api must be included via vendor-specific implementation.

Mail-sending Channel Adapter

Spring Integration provides support for outbound email with the MailSendingMessageHandler. It delegates to a configured instance of Spring’s JavaMailSender, as the following example shows:

 JavaMailSender mailSender = context.getBean("mailSender", JavaMailSender.class);

 MailSendingMessageHandler mailSendingHandler = new MailSendingMessageHandler(mailSender);

MailSendingMessageHandler has various mapping strategies that use Spring’s MailMessage abstraction. If the received message’s payload is already a MailMessage instance, it is sent directly. Therefore, we generally recommend that you precede this consumer with a transformer for non-trivial MailMessage construction requirements. However, Spring Integration supports a few simple message mapping strategies. For example, if the message payload is a byte array, that is mapped to an attachment. For simple text-based emails, you can provide a string-based message payload. In that case, a MailMessage is created with that String as the text content. If you work with a message payload type whose toString() method returns appropriate mail text content, consider adding Spring Integration’s ObjectToStringTransformer prior to the outbound mail adapter (see the example in Configuring a Transformer with XML for more detail).

You can also configure the outbound MailMessage with certain values from MessageHeaders. If available, values are mapped to the outbound mail’s properties, such as the recipients (To, Cc, and BCc), the from, the reply-to, and the subject. The header names are defined by the following constants:

 MailHeaders.SUBJECT
 MailHeaders.TO
 MailHeaders.CC
 MailHeaders.BCC
 MailHeaders.FROM
 MailHeaders.REPLY_TO

MailHeaders also lets you override corresponding MailMessage values. For example, if MailMessage.to is set to '[email protected]' and the MailHeaders.TO message header is provided, it takes precedence and overrides the corresponding value in MailMessage.

Mail-receiving Channel Adapter

Spring Integration also provides support for inbound email with the MailReceivingMessageSource. It delegates to a configured instance of Spring Integration’s own MailReceiver interface. There are two implementations: Pop3MailReceiver and ImapMailReceiver. The easiest way to instantiate either of these is by passing the 'uri' for a mail store to the receiver’s constructor, as the following example shows:

MailReceiver receiver = new Pop3MailReceiver("pop3://usr:pwd@localhost/INBOX");

Another option for receiving mail is the IMAP idle command (if supported by your mail server). Spring Integration provides the ImapIdleChannelAdapter, which is itself a message-producing endpoint. It delegates to an instance of the ImapMailReceiver but enables asynchronous reception of mail messages. The next section has examples of configuring both types of inbound channel adapter with Spring Integration’s namespace support in the 'mail' schema.

Normally, when the IMAPMessage.getContent() method is called, certain headers as well as the body are rendered (for a simple text email), as the following example shows:

To: [email protected]
From: [email protected]
Subject: Test Email

something

With a simple MimeMessage, getContent() returns the mail body (something in the preceding example).

Starting with version 2.2, the framework eagerly fetches IMAP messages and exposes them as an internal subclass of MimeMessage. This had the undesired side effect of changing the getContent() behavior. This inconsistency was further exacerbated by the Mail Mapping enhancement introduced in version 4.3, because, when a header mapper was provided, the payload was rendered by the IMAPMessage.getContent() method. This meant that the IMAP content differed, depending on whether or not a header mapper was provided.

Starting with version 5.0, messages originating from an IMAP source render the content in accordance with IMAPMessage.getContent() behavior, regardless of whether a header mapper is provided. If you do not use a header mapper and you wish to revert to the previous behavior of rendering only the body, set the simpleContent boolean property on the mail receiver to true. This property now controls the rendering regardless of whether a header mapper is used. It now allows body-only rendering when a header mapper is provided.

Starting with version 5.2, the autoCloseFolder option is provided on the mail receiver. Setting it to false doesn’t close the folder automatically after a fetch, but instead an IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE header (see MessageHeaderAccessor API for more information) is populated into every message to producer from the channel adapter. It is the target application’s responsibility to call the close() on this header whenever it is necessary in the downstream flow:

Closeable closeableResource = StaticMessageHeaderAccessor.getCloseableResource(mailMessage);
if (closeableResource != null) {
    closeableResource.close();
}

Keeping the folder open is useful in cases where communication with the server is needed during parsing multipart content of the email with attachments. The close() on the IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE header delegates to the AbstractMailReceiver to close the folder with expunge option if shouldDeleteMessages is configured respectively on the AbstractMailReceiver.

Inbound Mail Message Mapping

By default, the payload of messages produced by the inbound adapters is the raw MimeMessage. You can use that object to interrogate the headers and content. Starting with version 4.3, you can provide a HeaderMapper<MimeMessage> to map the headers to MessageHeaders. For convenience, Spring Integration provides a DefaultMailHeaderMapper for this purpose. It maps the following headers:

mail_from: A String representation of the from address.
mail_bcc: A String array containing the bcc addresses.
mail_cc: A String array containing the cc addresses.
mail_to: A String array containing the to addresses.
mail_replyTo: A String representation of the replyTo address.
mail_subject: The mail subject.
mail_lineCount: A line count (if available).
mail_receivedDate: The received date (if available).
mail_size: The mail size (if available).
mail_expunged: A boolean indicating if the message is expunged.
mail_raw: A MultiValueMap containing all the mail headers and their values.
mail_contentType: The content type of the original mail message.
contentType: The payload content type (see below).

When message mapping is enabled, the payload depends on the mail message and its implementation. Email contents are usually rendered by a DataHandler within the MimeMessage.

For a text/* email, the payload is a String and the contentType header is the same as mail_contentType.

For a messages with embedded javax.mail.Part instances, the DataHandler usually renders a Part object. These objects are not Serializable and are not suitable for serialization with alternative technologies such as Kryo. For this reason, by default, when mapping is enabled, such payloads are rendered as a raw byte[] containing the Part data. Examples of Part are Message and Multipart. The contentType header is application/octet-stream in this case. To change this behavior and receive a Multipart object payload, set embeddedPartsAsBytes to false on MailReceiver. For content types that are unknown to the DataHandler, the contents are rendered as a byte[] with a contentType header of application/octet-stream.

When you do not provide a header mapper, the message payload is the MimeMessage presented by javax.mail. The framework provides a MailToStringTransformer that you can use to convert the message by using a strategy to convert the mail contents to a String. This is also available by using the XML namespace, as the following example shows:

<int-mail:mail-to-string-transformer ... >

The following example does the same thing with Java configuration:

@Bean
@Transformer(inputChannel="...", outputChannel="...")
public Transformer transformer() {
    return new MailToStringTransformer();
}

The following example does the same thing with the Java DSL:

   ...
   .transform(Mail.toStringTransformer())
   ...

Starting with version 4.3, the transformer handles embedded Part instances (as well as Multipart instances, which were handled previously). The transformer is a subclass of AbstractMailTransformer that maps the address and subject headers from the preceding list. If you wish to perform some other transformation on the message, consider subclassing AbstractMailTransformer.

Mail Namespace Support

Spring Integration provides a namespace for mail-related configuration. To use it, configure the following schema locations:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:int-mail="http://www.springframework.org/schema/integration/mail"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
    https://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/integration/mail
    https://www.springframework.org/schema/integration/mail/spring-integration-mail.xsd">

To configure an outbound channel adapter, provide the channel from which to receive and the MailSender, as the following example shows:

<int-mail:outbound-channel-adapter channel="outboundMail"
    mail-sender="mailSender"/>

Alternatively, you can provide the host, username, and password, as the following example shows:

<int-mail:outbound-channel-adapter channel="outboundMail"
    host="somehost" username="someuser" password="somepassword"/>

Starting with version 5.1.3, the host, username ane mail-sender can be omitted, if java-mail-properties is provided. However the host and username has to be configured with appropriate Java mail properties, e.g. for SMTP:

[email protected]
mail.smtp.host=smtp.gmail.com
mail.smtp.port=587

As with any outbound Channel Adapter, if the referenced channel is a PollableChannel, you should provide a <poller> element (see Endpoint Namespace Support).

When you use the namespace support, you can also use a header-enricher message transformer. Doing so simplifies the application of the headers mentioned earlier to any message prior to sending to the mail outbound channel adapter.

The following example assumes the payload is a Java bean with appropriate getters for the specified properties, but you can use any SpEL expression:

<int-mail:header-enricher input-channel="expressionsInput" default-overwrite="false">
	<int-mail:to expression="payload.to"/>
	<int-mail:cc expression="payload.cc"/>
	<int-mail:bcc expression="payload.bcc"/>
	<int-mail:from expression="payload.from"/>
	<int-mail:reply-to expression="payload.replyTo"/>
	<int-mail:subject expression="payload.subject" overwrite="true"/>
</int-mail:header-enricher>

Alternatively, you can use the value attribute to specify a literal. You also can specify default-overwrite and individual overwrite attributes to control the behavior with existing headers.

To configure an inbound channel adapter, you have the choice between polling or event-driven (assuming your mail server supports IMAP idle — if not, then polling is the only option). A polling channel adapter requires the store URI and the channel to which to send inbound messages. The URI may begin with pop3 or imap. The following example uses an imap URI:

<int-mail:inbound-channel-adapter id="imapAdapter"
      store-uri="imaps://[username]:[password]@imap.gmail.com/INBOX"
      java-mail-properties="javaMailProperties"
      channel="receiveChannel"
      should-delete-messages="true"
      should-mark-messages-as-read="true"
      auto-startup="true">
      <int:poller max-messages-per-poll="1" fixed-rate="5000"/>
</int-mail:inbound-channel-adapter>

If you do have IMAP idle support, you may want to configure the imap-idle-channel-adapter element instead. Since the idle command enables event-driven notifications, no poller is necessary for this adapter. It sends a message to the specified channel as soon as it receives the notification that new mail is available. The following example configures an IMAP idle mail channel:

<int-mail:imap-idle-channel-adapter id="customAdapter"
      store-uri="imaps://[username]:[password]@imap.gmail.com/INBOX"
      channel="receiveChannel"
      auto-startup="true"
      should-delete-messages="false"
      should-mark-messages-as-read="true"
      java-mail-properties="javaMailProperties"/>

You can provide javaMailProperties by creating and populating a regular java.utils.Properties object — for example, by using the util namespace provided by Spring.

If your username contains the '@' character, use '%40' instead of '@' to avoid parsing errors from the underlying JavaMail API.

The following example shows how to configure a java.util.Properties object:

<util:properties id="javaMailProperties">
  <prop key="mail.imap.socketFactory.class">javax.net.ssl.SSLSocketFactory</prop>
  <prop key="mail.imap.socketFactory.fallback">false</prop>
  <prop key="mail.store.protocol">imaps</prop>
  <prop key="mail.debug">false</prop>
</util:properties>

By default, the ImapMailReceiver searches for messages based on the default SearchTerm, which is all mail messages that:

Are RECENT (if supported)
Are NOT ANSWERED
Are NOT DELETED
Are NOT SEEN
hHave not been processed by this mail receiver (enabled by the use of the custom USER flag or simply NOT FLAGGED if not supported)

The custom user flag is spring-integration-mail-adapter, but you can configure it. Since version 2.2, the SearchTerm used by the ImapMailReceiver is fully configurable with SearchTermStrategy, which you can inject by using the search-term-strategy attribute. A SearchTermStrategy is a strategy interface with a single method that lets you create an instance of the SearchTerm used by the ImapMailReceiver. The following listing shows the SearchTermStrategy interface:

public interface SearchTermStrategy {

    SearchTerm generateSearchTerm(Flags supportedFlags, Folder folder);

}

The following example relies TestSearchTermStrategy rather than the default SearchTermStrategy:

<mail:imap-idle-channel-adapter id="customAdapter"
			store-uri="imap:something"
			…
			search-term-strategy="searchTermStrategy"/>

<bean id="searchTermStrategy"
  class="o.s.i.mail.config.ImapIdleChannelAdapterParserTests.TestSearchTermStrategy"/>

See Marking IMAP Messages When \Recent Is Not Supported for information about message flagging.

Important: IMAP PEEK

Starting with version 4.1.1, the IMAP mail receiver uses the mail.imap.peek or mail.imaps.peek JavaMail property, if specified. Previously, the receiver ignored the property and always set the PEEK flag. Now, if you explicitly set this property to false, the message ise marked as \Seen regardless of the setting of shouldMarkMessagesRead. If not specified, the previous behavior is retained (peek is true).

IMAP `idle` and Lost Connections

When using an IMAP idle channel adapter, connections to the server may be lost (for example, through network failure) and, since the JavaMail documentation explicitly states that the actual IMAP API is experimental, it is important to understand the differences in the API and how to deal with them when configuring IMAP idle adapters. Currently, Spring Integration mail adapters were tested with JavaMail 1.4.1 and JavaMail 1.4.3. Depending on which one is used, you must pay special attention to some of the JavaMail properties that need to be set with regard to auto-reconnect.

The following behavior was observed with Gmail but should provide you with some tips on how to solve re-connect issue with other providers. However feedback is always welcome. Again, the following notes are based on Gmail.

With JavaMail 1.4.1, if you set the mail.imaps.timeout property to a relatively short period of time (approximately 5 min in our testing), IMAPFolder.idle() throws FolderClosedException after this timeout. However, if this property is not set (it should be indefinite) the IMAPFolder.idle() method never returns and never throws an exception. It does, however, reconnect automatically if the connection was lost for a short period of time (under 10 min in our testing). However, if the connection was lost for a long period of time (over 10 min), IMAPFolder.idle(), does not throw FolderClosedException and does not re-establish the connection, and remains in the blocked state indefinitely, thus leaving you no possibility to reconnect without restarting the adapter. Consequently, the only way to make re-connecting work with JavaMail 1.4.1 is to set the mail.imaps.timeout property explicitly to some value, but it also means that such value should be relatively short (under 10 min) and the connection should be re-established relatively quickly. Again, it may be different with providers other than Gmail. With JavaMail 1.4.3 introduced significant improvements to the API, ensuring that there is always a condition that forces the IMAPFolder.idle() method to return StoreClosedException or FolderClosedException or to simply return, thus letting you proceed with auto-reconnecting. Currently auto-reconnecting runs infinitely making attempts to reconnect every ten seconds.

In both configurations, channel and should-delete-messages are required attributes. You should understand why should-delete-messages is required. The issue is with the POP3 protocol, which does not have any knowledge of messages that were read. It can only know what has been read within a single session. This means that, when your POP3 mail adapter runs, emails are successfully consumed as as they become available during each poll and no single email message is delivered more then once. However, as soon as you restart your adapter and begin a new session, all the email messages that might have been retrieved in the previous session are retrieved again. That is the nature of POP3. Some might argue that should-delete-messages should be true by default. In other words, there are two valid and mutually exclusive use that make it very hard to pick a single best default. You may want to configure your adapter as the only email receiver, in which case you want to be able to restart your adapter without fear that previously delivered messages are not delivered again. In this case, setting should-delete-messages to true would make the most sense. However, you may have another use case where you may want to have multiple adapters monitor email servers and their content. In other words, you want to 'peek but not touch'. Then setting should-delete-messages to false is much more appropriate. So since it is hard to choose what should be the right default value for the should-delete-messages attribute, we made it a required attribute to be set by you. Leaving it up to you also means that you are less likely to end up with unintended behavior.

When configuring a polling email adapter’s should-mark-messages-as-read attribute, you should be aware of the protocol you are configuring to retrieve messages. For example, POP3 does not support this flag, which means setting it to either value has no effect, as messages are not marked as read.

In the case of a silently dropped connection, an idle cancel task is run in the background periodically (a new IDLE will usually immediately be processed). To control this interval, a cancelIdleInterval option is provided; default 120 (2 minutes). RFC 2177 recommends an interval no larger than 29 minutes.

You should understand that that these actions (marking messages read and deleting messages) are performed after the messages are received but before they are processed. This can cause messages to be lost.

You may wish to consider using transaction synchronization instead. See Transaction Synchronization.

The <imap-idle-channel-adapter/> also accepts the 'error-channel' attribute. If a downstream exception is thrown and an 'error-channel' is specified, a MessagingException message containing the failed message and the original exception is sent to this channel. Otherwise, if the downstream channels are synchronous, any such exception is logged as a warning by the channel adapter.

Beginning with the 3.0 release, the IMAP idle adapter emits application events (specifically ImapIdleExceptionEvent instances) when exceptions occur. This allows applications to detect and act on those exceptions. You can obtain the events by using an <int-event:inbound-channel-adapter> or any ApplicationListener configured to receive an ImapIdleExceptionEvent or one of its super classes.

Marking IMAP Messages When `\Recent` Is Not Supported

If shouldMarkMessagesAsRead is true, the IMAP adapters set the \Seen flag.

In addition, when an email server does not support the \Recent flag, the IMAP adapters mark messages with a user flag (by default, spring-integration-mail-adapter), as long as the server supports user flags. If not, Flag.FLAGGED is set to true. These flags are applied regardless of the shouldMarkMessagesRead setting.

As discussed in [search-term], the default SearchTermStrategy ignore messages that are so flagged.

Starting with version 4.2.2, you can set the name of the user flag by using setUserFlag on the MailReceiver. Doing so lets multiple receivers use a different flag (as long as the mail server supports user flags). The user-flag attribute is available when configuring the adapter with the namespace.

Email Message Filtering

Very often, you may encounter a requirement to filter incoming messages (for example, you want to read only emails that have 'Spring Integration' in the Subject line). You can accomplish this by connecting an inbound mail adapter with an expression-based Filter. Although it would work, there is a downside to this approach. Since messages would be filtered after going through the inbound mail adapter, all such messages would be marked as read (SEEN) or unread (depending on the value of should-mark-messages-as-read attribute). However, in reality, it be more useful to mark messages as SEEN only if they pass the filtering criteria. This is similar to looking at your email client while scrolling through all the messages in the preview pane, but only flagging messages that were actually opened and read as SEEN.

Spring Integration 2.0.4 introduced the mail-filter-expression attribute on inbound-channel-adapter and imap-idle-channel-adapter. This attribute lets you provide an expression that is a combination of SpEL and a regular expression. For example if you would like to read only emails that contain 'Spring Integration' in the subject line, you would configure the mail-filter-expression attribute like as follows: mail-filter-expression="subject matches '(?i).Spring Integration.".

Since javax.mail.internet.MimeMessage is the root context of the SpEL evaluation context, you can filter on any value available through MimeMessage, including the actual body of the message. This one is particularly important, since reading the body of the message typically results in such messages being marked as SEEN by default. However, since we now set the PEEK flag of every incoming message to 'true', only messages that were explicitly marked as SEEN are marked as read.

So, in the following example, only messages that match the filter expression are output by this adapter and only those messages are marked as read:

<int-mail:imap-idle-channel-adapter id="customAdapter"
	store-uri="imaps://some_google_address:${password}@imap.gmail.com/INBOX"
	channel="receiveChannel"
	should-mark-messages-as-read="true"
	java-mail-properties="javaMailProperties"
	mail-filter-expression="subject matches '(?i).*Spring Integration.*'"/>

In the preceding example, thanks to the mail-filter-expression attribute, only messages that contain 'Spring Integration' in the subject line are produced by this adapter.

Another reasonable question is what happens on the next poll or idle event or what happens when such an adapter is restarted. Can there be duplication of massages to be filtered? In other words, if, on the last retrieval where you had five new messages and only one passed the filter, what would happen with the other four? Would they go through the filtering logic again on the next poll or idle? After all, they were not marked as SEEN. The answer is no. They would not be subject to duplicate processing due to another flag (RECENT) that is set by the email server and is used by the Spring Integration mail search filter. Folder implementations set this flag to indicate that this message is new to this folder. That is, it has arrived since the last time this folder was opened. In other words, while our adapter may peek at the email, it also lets the email server know that such email was touched and should therefore be marked as RECENT by the email server.

Transaction Synchronization

Transaction synchronization for inbound adapters lets you take different actions after a transaction commits or rolls back. You can enable transaction synchronization by adding a <transactional/> element to the poller for the polled <inbound-adapter/> or to the <imap-idle-inbound-adapter/>. Even if there is no 'real' transaction involved, you can still enable this feature by using a PseudoTransactionManager with the <transactional/> element. For more information, see Transaction Synchronization.

Because of the many different mail servers and specifically the limitations that some have, at this time we provide only a strategy for these transaction synchronizations. You can send the messages to some other Spring Integration components or invoke a custom bean to perform some action. For example, to move an IMAP message to a different folder after the transaction commits, you might use something similar to the following:

<int-mail:imap-idle-channel-adapter id="customAdapter"
    store-uri="imaps://something.com:[email protected]/INBOX"
    channel="receiveChannel"
    auto-startup="true"
    should-delete-messages="false"
    java-mail-properties="javaMailProperties">
    <int:transactional synchronization-factory="syncFactory"/>
</int-mail:imap-idle-channel-adapter>

<int:transaction-synchronization-factory id="syncFactory">
    <int:after-commit expression="@syncProcessor.process(payload)"/>
</int:transaction-synchronization-factory>

<bean id="syncProcessor" class="thing1.thing2.Mover"/>

The following example shows what the Mover class might look like:

public class Mover {

    public void process(MimeMessage message) throws Exception{
        Folder folder = message.getFolder();
        folder.open(Folder.READ_WRITE);
        String messageId = message.getMessageID();
        Message[] messages = folder.getMessages();
        FetchProfile contentsProfile = new FetchProfile();
        contentsProfile.add(FetchProfile.Item.ENVELOPE);
        contentsProfile.add(FetchProfile.Item.CONTENT_INFO);
        contentsProfile.add(FetchProfile.Item.FLAGS);
        folder.fetch(messages, contentsProfile);
        // find this message and mark for deletion
        for (int i = 0; i < messages.length; i++) {
            if (((MimeMessage) messages[i]).getMessageID().equals(messageId)) {
                messages[i].setFlag(Flags.Flag.DELETED, true);
                break;
            }
        }

        Folder somethingFolder = store.getFolder("SOMETHING"));
        somethingFolder.appendMessages(new MimeMessage[]{message});
        folder.expunge();
        folder.close(true);
        somethingFolder.close(false);
    }
}

For the message to be still available for manipulation after the transaction, should-delete-messages must be set to 'false'.

Configuring channel adapters with the Java DSL

To configure mail mail component in Java DSL, the framework provides a o.s.i.mail.dsl.Mail factory, which can be used like this:

@SpringBootApplication
public class MailApplication {

    public static void main(String[] args) {
        new SpringApplicationBuilder(MailApplication.class)
            .web(false)
            .run(args);
    }

    @Bean
    public IntegrationFlow imapMailFlow() {
        return IntegrationFlows
                .from(Mail.imapInboundAdapter("imap://user:pw@host:port/INBOX")
                            .searchTermStrategy(this::fromAndNotSeenTerm)
                            .userFlag("testSIUserFlag")
                            .simpleContent(true)
                            .javaMailProperties(p -> p.put("mail.debug", "false")),
                    e -> e.autoStartup(true)
                            .poller(p -> p.fixedDelay(1000)))
                .channel(MessageChannels.queue("imapChannel"))
                .get();
    }

    @Bean
    public IntegrationFlow sendMailFlow() {
        return IntegrationFlows.from("sendMailChannel")
                .enrichHeaders(Mail.headers()
                        .subjectFunction(m -> "foo")
                        .from("foo@bar")
                        .toFunction(m -> new String[] { "bar@baz" }))
                .handle(Mail.outboundAdapter("gmail")
                            .port(smtpServer.getPort())
                            .credentials("user", "pw")
                            .protocol("smtp")),
                    e -> e.id("sendMailEndpoint"))
                .get();
    }
}