3. Fast Connection Failover

Oracle's RAC (Real Application Clusters) is an option that supports deployment of a single database across a cluster of servers, providing fault tolerance from hardware failures or other outages. Since a single database is served by a number of nodes, any node failure can be detected and subsequent operations can be directed to other nodes in the cluster. This support is provided by the "Fast Connection Failover" feature (FCF). When the failover occurs the current transaction is rolled back and a new transaction has to be initiated.

Spring's FCF support detects the transaction failure and attempts to retry the entire transaction. If this retry is successful it means that the client of the failed application will be unaware of this failover and it will look like the transaction completed after a brief delay.

The configuration for the FCF support is a two step configuration. First you need to configure a DataSource for RAC and second you need to configure an AOP advisor with a failover interceptor to handle the retries.

3.1 DataSource Configuration

We are going to need a DataSource that is capable of participating in a "Fast Connection Failover" scenario. The only one we have available is the oracle.jdbc.pool.OracleDataSource implementation that we will configure using the "orcl" namespace. This DataSource configured with some additional properties used for RAC.

We will be using the following property file to specify the username and password for the following example.

username=spring
password=spring

The url used in this example is a two node RAC configuration using the thin driver. It is probably too long to fit on the screen or on the page so if you would like to see the entire url it's listed in the callout notes.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:orcl="http://www.springframework.org/schema/data/orcl"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/context
       http://www.springframework.org/schema/context/spring-context-3.0.xsd
       http://www.springframework.org/schema/data/orcl
       http://www.springframework.org/schema/data/orcl/spring-data-orcl-1.0.xsd">

    <orcl:pooling-datasource id="racDataSource"
        url="jdbc:oracle:thin:@(description=(address_list=
            (address=(host=rac1)(protocol=tcp)(port=1521))
            (address=(host=rac2)(protocol=tcp)(port=1521)))
            (connect_data=(service_name=racdb1)))"
        properties-location="classpath:orcl.properties"
        fast-connection-failover-enabled="true" 1
        ONS-configuration="rac1:6200,rac2:6200"/> 2

    <bean id="transactionManager" 
          class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="racDataSource"/>
    </bean>

</beans>

1

The fast connection failover is enabled here.

2

The ONS (Oracle Notification Service) configuration is defined here since we are using the thin driver for this example.

3.2 AOP Configuration for Fast Connection Failover Retry

In order for the Fast Connection Failover to be transparent to the end user you need to consider the overall impact of this failover. The original transaction will fail and another transaction will be started to retry the same operation. You need to consider any non-transactional side effects that the failed transaction might have caused. You also need to consider work done while the transaction is suspended. This could happen if a method with a transactional attribute of "REQUIRES_NEW" is executed within the original transaction.

Once you have considered any possible side effects, you can proceed to configure a RacFailoverInterceptor together with the AOP advisor and pointcut. The failover advisor must be before or at the same pointcut where the transaction advisor is applied. If the pointcuts for the failover advisor and the transaction advisor are at the same pointcut then the failover advisor must have a higher priority than the transaction advisor that it should wrap.

For the AOP advisor configuration we use the "aop" namespace and for the RacFailoverInterceptor we use the rac-failover-interceptor tag from the "orcl" namespace.

3.2.1 Configuration when defining transactions using a <tx:advice> and an <aop:advisor>

When using a <tx:advice> combined with an <aop:advisor> you simply add an additional <aop:advisor> for the RAC failover Interceptor referencing the <orcl:rac-failover-interceptor> element. You must make sure that the RAC failover interceptor comes before the transaction advice and you can do that by specifying the order attribute on the advisor for the RAC failover interceptor. Any advisor specified without an order automatically gets the lowest priority, so by specifying order="1" for the RAC failover interceptor we are assured this advice will come before the transaction advice.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:orcl="http://www.springframework.org/schema/data/orcl"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
       http://www.springframework.org/schema/tx
       http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/data/orcl
       http://www.springframework.org/schema/data/orcl/spring-data-orcl-1.0.xsd">

    <aop:config>
        <aop:advisor pointcut="execution(* *..PetStoreFacade.insertOrder(..))" 1 
            advice-ref="racFailoverInterceptor" order="1"/>
        <aop:advisor pointcut="execution(* *..PetStoreFacade.*(..))" 2 
            advice-ref="txAdvice"/>
    </aop:config>

    <orcl:rac-failover-interceptor id="racFailoverInterceptor"/> 3

    <tx:advice id="txAdvice">
        <tx:attributes>
            <tx:method name="insert*"/>
            <tx:method name="update*"/>
            <tx:method name="*" read-only="true"/>
        </tx:attributes>
    </tx:advice>

</beans>

1

The advisor defined for the RAC failover interceptor. This must have a higher order than the transaction advisor. We do use the same pointcut

2

The standard transaction advice defined here.

3

The RAC failover interceptor is defined using the rac-failover-interceptor element of the "orcl" namespace.

3.2.2 Configuration when defining transactions using @Transactional annotation

When using a <tx:annotation-driven> configuration you add <aop:config> entry with an <aop:advisor> element for the RAC failover Interceptor referencing the <orcl:rac-failover-interceptor> element. You must make sure that the RAC failover interceptor comes before the transaction advice and you can do that by specifying the order attribute on the advisor for the RAC failover interceptor. Any <tx:annotation-driven> specified without an order automatically gets the lowest priority, so by specifying order="1" for the RAC failover interceptor we are assured this advice will come before the transaction advice.

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:orcl="http://www.springframework.org/schema/data/orcl"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
       http://www.springframework.org/schema/aop
       http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
       http://www.springframework.org/schema/tx
       http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
       http://www.springframework.org/schema/data/orcl
       http://www.springframework.org/schema/data/orcl/spring-data-orcl-1.0.xsd">

    <aop:config>
        <aop:advisor 1
            pointcut="@annotation(org.springframework.transaction.annotation.Transactional)" 
            advice-ref="racFailoverInterceptor" order="1"/>
    </aop:config>

    <orcl:rac-failover-interceptor id="racFailoverInterceptor"/> 2

    <tx:annotation-driven/> 3

</beans>

3

The advisor defined for the RAC failover interceptor. This must have a higher order than the transaction advisor. We use an @annotation pointcut referencing the @Transactional annotation.

1

The RAC failover interceptor is defined using the rac-failover-interceptor element of the "orcl" namespace.

???

The standard transaction annotation-driven element defined here.

3.3 Configuration options for <rac-failover-interceptor>

There is a number of optional attributes you can use to configure the rac-failover-interceptor.

Table 3.1. <rac-failover-interceptor> attribute settings

AttributeRequiredDefaultDescription
recoverable-error-codesNo3113, 3114, 1033, 1034, 1089, 17002, 17008, 17410A comma separated list of integer error codes that will be used instead of the default set.
max-number-of-retriesNo5The maximum number of times the retry will be performed.
back-off-policyNoNoBackOffPolicyA specific back off policy to be used. This is a reference to a bean that implements BackOffPolicy *

* org.springframework.batch.retry.backoff.BackOffPolicy