[[new-in-2-1-0] == New in the 2.1 Release

Spring Data for Pivotal GemFire’s XML namespace supports full configuration of the Pivotal GemFire In-Memory Data Grid (IMDG). The XML namespace is one of two ways to configure Pivotal GemFire in a Spring context in order to properly manage Pivotal GemFire’s lifecycle inside the Spring container. The other way to configure Pivotal GemFire in a Spring context is by using annotation-based configuration.

While support for Pivotal GemFire’s native cache.xml persists for legacy reasons, Pivotal GemFire application developers who use XML configuration are encouraged to do everything in Spring XML to take advantage of the many wonderful things Spring has to offer, such as modular XML configuration, property placeholders and overrides, SpEL (Spring Expression Language), and environment profiles. Behind the XML namespace, Spring Data for Pivotal GemFire makes extensive use of Spring’s FactoryBean pattern to simplify the creation, configuration, and initialization of Pivotal GemFire components.

Pivotal GemFire provides several callback interfaces, such as CacheListener, CacheLoader, and CacheWriter, that let developers add custom event handlers. Using Spring’s IoC container, you can configure these callbacks as normal Spring beans and inject them into Pivotal GemFire components. This is a significant improvement over native cache.xml, which provides relatively limited configuration options and requires callbacks to implement Pivotal GemFire’s Declarable interface (see Wiring Declarable Components to see how you can still use Declarables within Spring’s container).

In addition, IDEs, such as the Spring Tool Suite (STS), provide excellent support for Spring XML namespaces, including code completion, pop-up annotations, and real time validation.

To simplify configuration, Spring Data for Pivotal GemFire provides a dedicated XML namespace for configuring core Pivotal GemFire components. It is possible to configure beans directly by using Spring’s standard <bean> definition. However, all bean properties are exposed through the XML namespace, so there is little benefit to using raw bean definitions.

To use the Spring Data for Pivotal GemFire XML namespace, declare it in your Spring XML configuration meta-data, as the following example shows:

In addition to the core XML namespace (gfe), Spring Data for Pivotal GemFire provides a data access XML namespace (gfe-data), which is primarily intended to simplify the development of Pivotal GemFire client applications. This namespace currently contains support for Pivotal GemFire Repositories and Function execution, as well as a <datasource> tag that offers a convenient way to connect to a Pivotal GemFire cluster.

To use Pivotal GemFire, you need to either create a new cache or connect to an existing one. With the current version of Pivotal GemFire, you can have only one open cache per VM (more strictly speaking, per ClassLoader). In most cases, the cache should only be created once.

A peer Cache with default configuration can be created with the following simple declaration:

During Spring container initialization, any ApplicationContext containing this cache definition registers a CacheFactoryBean that creates a Spring bean named gemfireCache, which references a Pivotal GemFire Cache instance. This bean refers to either an existing Cache or, if one does not already exist, a newly created one. Since no additional properties were specified, a newly created Cache applies the default cache configuration.

All Spring Data for Pivotal GemFire components that depend on the Cache respect this naming convention, so you need not explicitly declare the Cache dependency. If you prefer, you can make the dependency explicit by using the cache-ref attribute provided by various SDG XML namespace elements. Also, you can override the cache’s bean name using the id attribute, as follows:

A Pivotal GemFire Cache can be fully configured using Spring. However, Pivotal GemFire’s native XML configuration file, cache.xml, is also supported. For situations where the Pivotal GemFire cache needs to be configured natively, you can provide a reference to the Pivotal GemFire XML configuration file by using the cache-xml-location attribute, as follows:

In this example, if a cache needs to be created, it uses a file named cache.xml located in the classpath root to configure it.

In addition to referencing an external XML configuration file, you can also specify Pivotal GemFire System properties that use any of Spring’s Properties support features.

For example, you can use the properties element defined in the util namespace to define Properties directly or load properties from a properties file, as follows:

Using a properties file is recommended for externalizing environment-specific settings outside the application configuration.

A Region is required to store and retrieve data from the cache. org.apache.geode.cache.Region is an interface extending java.util.Map and enables basic data access using familiar key-value semantics. The Region interface is wired into application classes that require it so the actual Region type is decoupled from the programming model. Typically, each Region is associated with one domain object, similar to a table in a relational database.

Pivotal GemFire implements the following types of Regions:

For more information about the various Region types and their capabilities as well as configuration options, please refer to Pivotal GemFire’s documentation on Region Types.

Pivotal GemFire allows indexes (also sometimes pluralized as indices) to be created on Region data to improve the performance of OQL (Object Query Language) queries.

In Spring Data for Pivotal GemFire, indexes are declared with the index element, as the following example shows:

In Spring Data for Pivotal GemFire’s XML schema (also called the SDG XML namespace), index bean declarations are not bound to a Region, unlike Pivotal GemFire’s native cache.xml. Rather, they are top-level elements similar to <gfe:cache> element. This lets you declare any number of indexes on any Region, whether they were just created or already exist — a significant improvement over Pivotal GemFire’s native cache.xml format.

An Index must have a name. You can give the Index an explicit name by using the name attribute. Otherwise, the bean name (that is, the value of the id attribute) of the index bean definition is used as the Index name.

The expression and from clause form the main components of an Index, identifying the data to index (that is, the Region identified in the from clause) along with what criteria (that is, expression) is used to index the data. The expression should be based on what application domain object fields are used in the predicate of application-defined OQL queries used to query and look up the objects stored in the Region.

Consider the following example, which has a lastName property:

Now consider the following example, which has an application-defined SDG Repository to query for Customer objects:

The SDG Repository finder/query method results in the following OQL statement being generated and ran:

Therefore, you might want to create an Index with a statement similar to the following:

The from clause must refer to a valid, existing Region and is how an Index gets applied to a Region. This is not specific to Spring Data for Pivotal GemFire. It is a feature of Pivotal GemFire.

The Index type may be one of three enumerated values defined by Spring Data for Pivotal GemFire’s IndexType enumeration: FUNCTIONAL, HASH, and PRIMARY_KEY.

Each of the enumerated values corresponds to one of the QueryService create[|Key|Hash]Index methods invoked when the actual Index is to be created (or “defined” — you can find more on “defining” indexes in the next section). For instance, if the IndexType is PRIMARY_KEY, then the QueryService.createKeyIndex(..) is invoked to create a KEY Index.

The default is FUNCTIONAL and results in one of the QueryService.createIndex(..) methods being invoked. See the Spring Data for Pivotal GemFire XML schema for a full set of options.

For more information on indexing in Pivotal GemFire, see “Working with Indexes” in Pivotal GemFire’s User Guide.

Spring Data for Pivotal GemFire supports DiskStore configuration and creation through the disk-store element, as the following example shows:

DiskStore instances are used by Regions for file system persistent backup and overflow of evicted entries as well as persistent backup for WAN Gateways. Multiple Pivotal GemFire components may share the same DiskStore. Additionally, multiple file system directories may be defined for a single DiskStore, as shown in the preceding example.

See Pivotal GemFire’s documentation for a complete explanation of Persistence and Overflow and configuration options on DiskStore instances.

Spring Data for Pivotal GemFire supports cache and Region snapshots by using Pivotal GemFire’s Snapshot Service. The out-of-the-box Snapshot Service support offers several convenient features to simplify the use of Pivotal GemFire’s Cache and Region Snapshot Service APIs.

As the Pivotal GemFire documentation explains, snapshots let you save and subsequently reload the cached data later, which can be useful for moving data between environments, such as from production to a staging or test environment in order to reproduce data-related issues in a controlled context. You can combine Spring Data for Pivotal GemFire’s Snapshot Service support with Spring’s bean definition profiles to load snapshot data specific to the environment as necessary.

Spring Data for Pivotal GemFire’s support for Pivotal GemFire’s Snapshot Service begins with the <gfe-data:snapshot-service> element from the <gfe-data> XML namespace.

For example, you can define cache-wide snapshots to be loaded as well as saved by using a couple of snapshot imports and a data export definition, as follows:

You can define as many imports and exports as you like. You can define only imports or only exports. The file locations and directory paths can be absolute or relative to the Spring Data for Pivotal GemFire application, which is the JVM process’s working directory.

The preceding example is pretty simple, and the Snapshot Service defined in this case refers to the Pivotal GemFire cache instance with the default name of gemfireCache (as described in Configuring a Cache). If you name your cache bean definition something other than the default, you can use the cache-ref attribute to refer to the cache bean by name, as follows:

You can also define a Snapshot Service for a particular Region by specifying the region-ref attribute, as follows:

When the region-ref attribute is specified, Spring Data for Pivotal GemFire’s SnapshotServiceFactoryBean resolves the region-ref attribute value to a Region bean defined in the Spring container and creates a RegionSnapshotService. The snapshot import and export definitions function the same way. However, the location must refer to a file on an export.

Spring Data for Pivotal GemFire provides annotation support for implementing, registering and executing Pivotal GemFire Functions.

Spring Data for Pivotal GemFire also provides XML namespace support for registering Pivotal GemFire Functions for remote function execution.

See Pivotal GemFire’s documentation for more information on the Function execution framework.

Pivotal GemFire Functions are declared as Spring beans and must implement the org.apache.geode.cache.execute.Function interface or extend org.apache.geode.cache.execute.FunctionAdapter.

The namespace uses a familiar pattern to declare Functions, as the following example shows:

In the following example, GatewaySenders are configured for a PARTITION Region by adding child elements (gateway-sender and gateway-sender-ref) to the Region. A GatewaySender may register EventFilters and TransportFilters.

The following example also shows a sample configuration of an AsyncEventQueue, which must also be auto-wired into a Region (not shown):

On the other end of a GatewaySender is a corresponding GatewayReceiver to receive Gateway events. The GatewayReceiver may also be configured with EventFilters and TransportFilters, as follows:

See the Pivotal GemFire documentation for a detailed explanation of all the configuration options.

Pivotal GemFire can be difficult to setup and use correctly, given all the configuration properties and different configuration options:

Further complexity comes from the different supported topologies: * (client/server * P2P * WAN) * distributed system design patterns (such as shared-nothing architecture).

The annotation-based configuration model aims to simplify all this and more.

The annotation-based configuration model is an alternative to XML-based configuration using Spring Data for Pivotal GemFire’s XML namespace. With XML, you could use the gfe XML schema for configuration and the gfe-data XML schema for data access related concerns. See "Bootstrapping Pivotal GemFire with the Spring container" for more details.

Like Spring Boot, Spring Data for Pivotal GemFire’s annotation-based configuration model was designed as an opinionated, convention-over-configuration approach for using Pivotal GemFire. Indeed, this annotation-based configuration model was inspired by Spring Boot as well as several other Spring and Spring Data projects, collectively.

By following convention, all annotations provide reasonable and sensible defaults for all configuration attributes. The default value for a given annotation attribute directly corresponds to the default value provided in Pivotal GemFire for the same configuration property.

The intention is to let you enable a Pivotal GemFire feature or an embedded service by declaring the annotation on your Spring @Configuration or @SpringBootApplication class without needing to unnecessarily configure a large number of properties just to use the feature.

Again, getting started, quickly and as easily, is the primary objective.

However, the option to customize the configuration metadata and behavior of Pivotal GemFire is there if you need it, and Spring Data for Pivotal GemFire’s annotation-based configuration quietly backs away. You need only specify the configuration attributes you wish to adjust. Also, as we will see later in this document, there are several ways to configure a Pivotal GemFire feature or embedded service by using the annotations.

You can find all the new SDG Java Annotations in the org.springframework.data.gemfire.config.annotation package.

Like all Spring Boot applications that begin by annotating the application class with @SpringBootApplication, a Spring Boot application can easily become a Pivotal GemFire cache application by declaring any one of three main annotations:

These three annotations are the Spring application developer’s starting point when working with Pivotal GemFire.

To realize the intent behind these annotations, you must understand that there are two types of cache instances that can be created with Pivotal GemFire: a client cache or a peer cache.

You can configure a Spring Boot application as a Pivotal GemFire cache client with an instance of ClientCache, which can communicate with an existing cluster of Pivotal GemFire servers used to manage the application’s data. The client-server topology is the most common system architecture employed when using Pivotal GemFire and you can make your Spring Boot application a cache client, with a ClientCache instance, simply by annotating it with @ClientCacheApplication.

Alternatively, a Spring Boot application may be a peer member of a Pivotal GemFire cluster. That is, the application itself is just another server in a cluster of servers that manages data. The Spring Boot application creates an "embedded", peer Cache instance when you annotate your application class with @PeerCacheApplication.

By extension, a peer cache application may also serve as a CacheServer too, allowing cache clients to connect and perform data access operations on the server. This is accomplished by annotating the application class with @CacheServerApplication in place of @PeerCacheApplication, which creates a peer Cache instance along with the CacheServer.

By way of example, if you want to create a Spring Boot cache client application, start with the following:

Or, if you want to create a Spring Boot application with an embedded peer Cache instance, where your application will be a server and peer member of a cluster, or distributed system, formed by Pivotal GemFire, start with the following:

Alternatively, you can use the @CacheServerApplication annotation in place of @PeerCacheApplication to create both an embedded peer Cache instance and a CacheServer running on localhost, listening on the default cache server port, 40404, as follows:

There are multiple ways that a client can connect to and communicate with servers in a Pivotal GemFire cluster. The most common and recommended approach is to use Pivotal GemFire Locators.

By default, Pivotal GemFire sets up a "DEFAULT" Pool connected to a CacheServer running on localhost, listening on port 40404 when a ClientCache instance is created. A CacheServer listens on port 40404, accepting connections from all system NICs. You do not need to do anything special to use the client-server topology. Simply annotate your server-side Spring Boot application with @CacheServerApplication and your client-side Spring Boot application with @ClientCacheApplication, and you are ready to go.

If you prefer, you can even start your servers with Gfsh’s start server command. Your Spring Boot @ClientCacheApplication can still connect to the server regardless of how it was started. However, you may prefer to configure and start your servers by using the Spring Data for Pivotal GemFire approach since a properly annotated Spring Boot application class is far more intuitive and easier to debug.

As an application developer, you will no doubt want to customize the "DEFAULT" Pool set up by Pivotal GemFire to possibly connect to one or more Locators, as the following example demonstrates:

Along with the locators attribute, the @ClientCacheApplication annotation has a servers attribute as well. The servers attribute can be used to specify one or more nested @Server annotations that let the cache client connect directly to one or more servers, if necessary.

You can also configure additional Pool instances (other than the "DEFAULT" Pool provided by Pivotal GemFire when a ClientCache instance is created with the @ClientCacheApplication annotation) by using the @EnablePool and @EnablePools annotations.

The following example uses the @EnablePool and @EnablePools annotations:

The name attribute is the only required attribute of the @EnablePool annotation. As we will see later, the value of the name attribute corresponds to both the name of the Pool bean created in the Spring container as well as the name used to reference the corresponding configuration properties. It is also the name of the Pool registered and used by Pivotal GemFire.

Similarly, on the server, you can configure multiple CacheServers that a client can connect to, as follows:

One thing an observant reader may have noticed is that, in all cases, you have specified hard-coded values for all hostnames, ports, and configuration-oriented annotation attributes. This is not ideal when the application gets promoted and deployed to different environments, such as from DEV to QA to STAGING to PROD.

The next section covers how to handle dynamic configuration determined at runtime.

Another goal when designing the annotation-based configuration model was to preserve type safety in the annotation attributes. For example, if the configuration attribute could be expressed as an int (such as a port number), then the attribute’s type should be an int.

Unfortunately, this is not conducive to dynamic and resolvable configuration at runtime.

One of the finer features of Spring is the ability to use property placeholders and SpEL expressions in properties or attributes of the configuration metadata when configuring beans in the Spring container. However, this would require all annotation attributes to be of type String, thereby giving up type safety, which is not desirable.

So, Spring Data for Pivotal GemFire borrows from another commonly used pattern in Spring, Configurers. Many different Configurer interfaces are provided in Spring Web MVC, including the org.springframework.web.servlet.config.annotation.ContentNegotiationConfigurer.

The Configurers design pattern enables application developers to receive a callback to customize the configuration of a component or bean on startup. The framework calls back to user-provided code to adjust the configuration at runtime. One of the more common uses of this pattern is to supply conditional configuration based on the application’s runtime environment.

Spring Data for Pivotal GemFire provides several Configurer callback interfaces to customize different aspects of the annotation-based configuration metadata at runtime, before the Spring managed beans that the annotations create are initialized:

For example, you can use the CacheServerConfigurer and ClientCacheConfigurer to customize the port numbers used by your Spring Boot CacheServer and ClientCache applications, respectively.

Consider the following example from a server application:

Next, consider the following example from a client application:

By using the provided Configurers, you can receive a callback to further customize the configuration that is enabled by the associated annotation at runtime, during startup.

In addition, when the Configurer is declared as a bean in the Spring container, the bean definition can take advantage of other Spring container features, such as property placeholders, SpEL expressions by using the @Value annotation on factory method parameters, and so on.

All Configurers provided by Spring Data for Pivotal GemFire take two bits of information in the callback: the name of the bean created in the Spring container by the annotation and a reference to the FactoryBean used by the annotation to create and configure the Pivotal GemFire component (for example, a ClientCache instance is created and configured with ClientCacheFactoryBean).

Given that a Configurer can be declared as a regular bean definition like any other POJO, you can combine different Spring configuration options, such as the use of Spring Profiles with Conditions that use both property placeholders and SpEL expressions. These and other nifty features let you create even more sophisticated and flexible configurations.

However, Configurers are not the only option.

In addition to Configurers, each annotation attribute in the annotation-based configuration model is associated with a corresponding configuration property (prefixed with spring.data.gemfire.), which can be declared in a Spring Boot application.properties file.

Building on the earlier examples, the client’s application.properties file would define the following set of properties:

The corresponding server’s application.properties file would define the following properties:

Then you can simplify the @ClientCacheApplication class to the following:

Also, the @CacheServerApplication class becomes the following:

The preceding example shows why it is important to "name" your annotation-based beans (other than because it is required in certain cases). Doing so makes it possible to reference the bean in a Spring context from XML, properties, and Java. It is even possible to inject annotation-defined beans into an application class, for whatever purpose, as the following example demonstrates:

Likewise, naming an annotation-defined bean lets you code a Configurer to customize a specific, "named" bean since the beanName is 1 of 2 arguments passed to the callback.

Oftentimes, an associated annotation attribute property takes two forms: a "named" property along with an "unnamed" property.

The following example shows such an arrangement:

While there are three named CacheServers above, there is one unnamed CacheServer property providing the default value for any unspecified value for that property, even for "named" CacheServers. So, while "Venus" sets and overrides its own bind-address, "Saturn" and "Neptune" inherit from the "unnamed" spring.data.gemfire.cache.server.bind-address property.

See an annotation’s Javadoc for which annotation attributes support property-based configuration and whether they support "named" properties over default, "unnamed" properties.

Pivotal GemFire provides the ability to start many different embedded services that are required by an application, depending on the use case.

Oftentimes, it is necessary to turn up logging in order to understand exactly what Pivotal GemFire is doing and when.

To enable Logging, annotate your application class with @EnableLogging and set the appropriate attributes or associated properties, as follows:

While the logLevel attribute can be specified with all the cache-based application annotations (for example, @ClientCacheApplication(logLevel="info")), it is easier to customize logging behavior with the @EnableLogging annotation.

Additionally, you can configure the log-level by setting the spring.data.gemfire.logging.level property in application.properties.

See the @EnableLogging annotation Javadoc for more details.

To gain even deeper insight into Pivotal GemFire at runtime, you can enable statistics. Gathering statistical data facilitates system analysis and troubleshooting when complex problems, which are often distributed in nature and where timing is a crucial factor, occur.

When statistics are enabled, you can use Pivotal GemFire’s VSD (Visual Statistics Display) tool to analyze the statistical data that is collected.

To enable statistics, annotate your application class with @EnableStatistics, as follows:

Enabling statistics on a server is particularly valuable when evaluating performance. To do so, annotate your @PeerCacheApplication or @CacheServerApplication class with @EnableStatistics.

You can use the @EnableStatistics annotation attributes or associated properties to customize the statistics gathering and collection process.

See the @EnableStatistics annotation Javadoc for more details.

More details on Pivotal GemFire’s statistics can be found here.

One of the more powerful features of Pivotal GemFire is PDX serialization. While a complete discussion of PDX is beyond the scope of this document, serialization using PDX is a much better alternative to Java serialization, with the following benefits:

In general, serialization in Pivotal GemFire is required any time data is transferred to or from clients and servers or between peers in a cluster during normal distribution and replication processes as well as when data is overflowed or persisted to disk.

Enabling PDX serialization is much simpler than modifying all of your application domain object types to implement java.io.Serializable, especially when it may be undesirable to impose such restrictions on your application domain model or you do not have any control over the objects your are serializing, which is especially true when using a 3rd party library (e.g. think of a geo-spatial API with Coordinate types).

To enable PDX, annotate your application class with @EnablePdx, as follows:

Typically, an application’s domain object types either implements the org.apache.geode.pdx.PdxSerializable interface or you can implement and register a non-invasive implementation of the org.apache.geode.pdx.PdxSerializer interface to handle all the application domain object types that need to be serialized.

Unfortunately, Pivotal GemFire only lets one PdxSerializer be registered, which suggests that all application domain object types need to be handled by a single PdxSerializer instance. However, that is a serious anti-pattern and an unmaintainable practice.

Even though only a single PdxSerializer instance can be registered with Pivotal GemFire, it makes sense to create a single PdxSerializer implementation per application domain object type.

By using the Composite Software Design Pattern, you can provide an implementation of the PdxSerializer interface that aggregates all of the application domain object type-specific PdxSerializer instances, but acts as a single PdxSerializer instance and register it.

You can declare this composite PdxSerializer as a managed bean in the Spring container and refer to this composite PdxSerializer by its bean name in the @EnablePdx annotation using the serializerBeanName attribute. Spring Data for Pivotal GemFire takes care of registering it with Pivotal GemFire on your behalf.

The following example shows how to create a custom composite PdxSerializer:

It is also possible to declare Pivotal GemFire’s org.apache.geode.pdx.ReflectionBasedAutoSerializer as a bean definition in a Spring context.

Alternatively, you should use Spring Data for Pivotal GemFire’s more robust org.springframework.data.gemfire.mapping.MappingPdxSerializer, which uses Spring Data mapping metadata and infrastructure applied to the serialization process for more efficient handling than reflection alone.

Many other aspects and features of PDX can be adjusted with the @EnablePdx annotation attributes or associated configuration properties.

See the @EnablePdx annotation Javadoc for more details.

Equally important to serializing data to be transferred over the wire is securing the data while in transit. Of course, the common way to accomplish this in Java is by using the Secure Sockets Extension (SSE) and Transport Layer Security (TLS).

To enable SSL, annotate your application class with @EnableSsl, as follows:

Then you need to set the necessary SSL configuration attributes or properties: keystores, usernames/passwords, and so on.

You can individually configure different Pivotal GemFire components (GATEWAY, HTTP, JMX, LOCATOR, and SERVER) with SSL, or you can collectively configure them to use SSL by using the CLUSTER enumerated value.

You can specify which Pivotal GemFire components the SSL configuration settings should applied by using the nested @EnableSsl annotation, components attribute with enumerated values from the Component enum, as follows:

In addition, you can also specify component-level SSL configuration (ciphers, protocols and keystore/truststore information) by using the corresponding annotation attribute or associated configuration properties.

See the @EnableSsl annotation Javadoc for more details.

More details on Pivotal GemFire SSL support can be found here.

While many of the gemfire.properties are conveniently encapsulated in and abstracted with an annotation in the SDG annotation-based configuration model, the less commonly used Pivotal GemFire properties are still accessible from the @EnableGemFireProperties annotation.

Annotating your application class with @EnableGemFireProperties is convenient and a nice alternative to creating a gemfire.properties file or setting Pivotal GemFire properties as Java system properties on the command line when launching your application.

A few examples of some of the less common Pivotal GemFire properties that you usually need not worry about include, but are not limited to: ack-wait-threshold, disable-tcp, socket-buffer-size, and others.

To individually set any Pivotal GemFire property, annotate your application class with @EnableGemFireProperties and set the Pivotal GemFire properties you want to change from the default value set by Pivotal GemFire with the corresponding attribute, as follows:

Keep in mind that some of the Pivotal GemFire properties are client-specific (for example, conflateEvents), while others are server-specific (for example distributedSystemId, enableNetworkPartitionDetection, enforceUniqueHost, memberTimeout, redundancyZone, and others).

More details on Pivotal GemFire properties can be found here.

So far, outside of PDX, our discussion has centered around configuring Pivotal GemFire’s more administrative functions: creating a cache instance, starting embedded services, enabling logging, statistics, SSL, and using gemfire.properties to affect low-level configuration and behavior. While all these configuration options are important, none of them relate directly to your application. In other words, we still need some place to store our application data and make it generally available and accessible.

Pivotal GemFire organizes data in a cache into Regions. You can think of a Region as a table in a relational database. Generally, a Region should only store a single type of object, which makes it more conducive for building effective indexes and writing queries. We cover indexing later.

Previously, Spring Data for Pivotal GemFire users needed to explicitly define and declare the Regions used in their applications to store data by writing very verbose Spring configuration metadata, whether using SDG’s FactoryBeans from the API in Spring’s Java-based container configuration or using XML.

The following example demonstrates how to configure a Region bean in Java:

The following example shows how to configure the same Region bean in XML:

While neither Java nor XML configuration is all that difficult to specify, either one can be cumbersome, especially if an application requires a large number of Regions. Many relational database-based applications can literally have hundreds or even thousands of tables.

Now you can define and configure Regions based on their application domain objects (entities) themselves. No longer do you need to explicitly define Region bean definitions in Spring configuration metadata, unless you require finer-grained control.

To simplify Region creation, Spring Data for Pivotal GemFire combines the use of Spring Data Repositories with the expressive power of annotation-based configuration using the new @EnableEntityDefinedRegions annotation.

First, an application developer starts by defining the application’s domain objects, as follows:

Next, you define a basic repository for Books by extending Spring Data Commons org.springframework.data.repository.CrudRepository interface, as follows:

The org.springframe.data.repository.CrudRepository is a Data Access Object (DAO) providing basic data access operations (CRUD) along with support for simple queries (such as findById(..)). You can define additional, more sophisticated queries by declaring query methods on the repository interface (for example, List<BooK> findByAuthor(Author author);).

Under the hood, Spring Data for Pivotal GemFire provides an implementation of your applications repository interfaces when the Spring container is bootstrapped. SDG even implements the query methods you define so long as you follow the conventions.

Now, when you defined the Book class, you also specified the Region in which instances of Book are mapped (stored) by declaring the Spring Data for Pivotal GemFire mapping annotation, @Region on the entity’s type. Of course, if the entity type (Book, in this case) referenced in the type parameter of the repository interface (BookRepository, in this case) is not annotated with @Region, the name is derived from the simple class name of the entity type (also Book, in this case).

Spring Data for Pivotal GemFire uses the mapping context, which contains mapping metadata for all the entities defined in your application, to determine all the Regions that are needed at runtime.

To enable and use this feature, annotate the application class with @EnableEntityDefinedRegions, as follows:

By default, the @EnableEntityDefinedRegions annotation scans for entity classes recursively, starting from the package of the configuration class on which the @EnableEntityDefinedRegions annotation is declared.

However, it is common to limit the search during the scan by setting the basePackages attribute with the package names containing your application entity classes.

Alternatively, you can use the more type-safe basePackageClasses attribute for specifying the package to scan by setting the attribute to an entity type in the package that contains the entity’s class, or by using a non-entity placeholder class specifically created for identifying the package to scan.

The following example shows how to specify the entity types to scan:

In addition to specifying where to begin the scan, like Spring’s @ComponentScan annotation, you can specify include and exclude filters with all the same semantics of the org.springframework.context.annotation.ComponentScan.Filter annotation.

See the @EnableEntityDefinedRegions annotation Javadoc for more details.

Another very important and useful feature of Pivotal GemFire is Continuous Queries.

In a world of Internet-enabled things, events and streams of data come from everywhere. Being able to handle and process a large stream of data and react to events in real time is an increasingly important requirement for many applications. One example is self-driving vehicles. Being able to receive, filter, transform, analyze, and act on data in real time is a key differentiator and characteristic of real time applications.

Fortunately, Pivotal GemFire was ahead of its time in this regard. By using Continuous Queries (CQ), a client application can express the data or events it is interested in and register listeners to handle and process the events as they occur. The data that a client application may be interested in is expressed as an OQL query, where the query predicate is used to filter or identify the data of interest. When data is changed or added and it matches the criteria defined in the query predicate of the registered CQ, the client application is notified.

Spring Data for Pivotal GemFire makes it easy to define and register CQs, along with an associated listener to handle and process CQ events without all the cruft of Pivotal GemFire’s plumbing. SDG’s new annotation-based configuration for CQs builds on the existing Continuous Query support in the continuous query listener container.

For instance, say a book publisher wants to register interest in and receive notification any time orders (demand) for a Book exceeds the current inventory (supply). Then the publisher’s print application might register the following CQ:

To enable Continuous Queries, annotate your application class with @EnableContinuousQueries.

Defining Continuous Queries consists of annotating any Spring @Component-annotated POJO class methods with the @ContinuousQuery annotation (in similar fashion to SDG’s Function-annotated POJO methods). A POJO method defined with a CQ by using the @ContinuousQuery annotation is called any time data matching the query predicate is added or changed.

Additionally, the POJO method signature should adhere to the requirements outlined in the section on the ContinuousQueryListener and the ContinuousQueryListenerAdapter.

See the @EnableContinuousQueries and @ContinuousQuery annotation Javadoc for more details on available attributes and configuration settings.

More details on Spring Data for Pivotal GemFire’s continuous query support can be found here.

More details on Pivotal GemFire’s Continuous Queries can be found here.

With Spring Data for Pivotal GemFire, Pivotal GemFire can be used as a caching provider in Spring’s cache abstraction.

In Spring’s Cache Abstraction, the caching annotations (such as @Cacheable) identify the cache on which a cache lookup is performed before invoking a potentially expensive operation. The results of an application service method are cached after the operation is invoked.

In Spring Data for Pivotal GemFire, a Spring Cache corresponds directly to a Pivotal GemFire Region. The Region must exist before any caching annotated application service methods are called. This is true for any of Spring’s caching annotations (that is, @Cacheable, @CachePut and @CacheEvict) that identify the cache to use in the service operation.

For instance, our publisher’s Point-of-Sale (PoS) application might have a feature to determine or lookup the Price of a Book during a sales transaction, as the following example shows:

To make your work easier when you use Spring Data for Pivotal GemFire with Spring’s Cache Abstraction, two new features have been added to the annotation-based configuration model.

Consider the following Spring caching configuration:

Using Spring Data for Pivotal GemFire’s new features, you can simplify the same caching configuration to the following:

First, the @EnableGemfireCaching annotation replaces both the Spring @EnableCaching annotation and the need to declare an explicit CacheManager bean definition (named "cacheManager") in the Spring config.

Second, the @EnableCachingDefinedRegions annotation, like the @EnableEntityDefinedRegions annotation described in “Configuring Regions”, inspects the entire Spring application, caching annotated service components to identify all the caches that are needed by the application at runtime and creates Regions in Pivotal GemFire for these caches on application startup.

The Regions created are local to the application process that created the Regions. If the application is a peer Cache, the Regions exist only on the application node. If the application is a ClientCache, then SDG creates client PROXY Regions and expects those Regions with the same name to already exist on the servers in the cluster.

Refer to the “Support for the Spring Cache Abstraction” section for more details on using Pivotal GemFire as a caching provider in Spring’s Cache Abstraction.

More details on Spring’s Cache Abstraction can be found here.

This may be the most exciting new feature in Spring Data for Pivotal GemFire.

When a client application class is annotated with @EnableClusterConfiguration, any Regions or indexes defined and declared as beans in the Spring container by the client application are “pushed” to the cluster of servers to which the client is connected. Not only that, but this “push” is performed in such a way that Pivotal GemFire remembers the configuration pushed by the client when using HTTP. If all the nodes in the cluster go down, they come back up with the same configuration as before. If a new server is added to the cluster, it will acquire identical configuration.

In a sense, this feature is not much different than if you were to use Gfsh to manually create the Regions and indexes on all the servers in the cluster. Except that now, with Spring Data for Pivotal GemFire, you no longer need to use Gfsh to create Regions and indexes. Your Spring Boot application, enabled with the power of Spring Data for Pivotal GemFire, already contains all the configuration metadata needed to create Regions and indexes for you.

When you use the Spring Data Repository abstraction, we know all the Regions (such as those defined by the @Region annotated entity classes) and indexes (such as those defined by the @Indexed-annotated entity fields and properties) that your application will need.

When you use Spring’s Cache Abstraction, we also know all the Regions for all the caches identified in the caching annotations needed by the application’s service components.

Essentially, you are already telling us everything we need to know simply by developing your application with the Spring Framework by using all of its provided services, infrastructure, and other components, whether expressed in annotation metadata, Java, XML or otherwise, and whether for configuration, mapping, or whatever the purpose.

The point is that you can focus on your application’s business logic while using the framework’s services and supporting infrastructure (such as Spring’s Cache Abstraction, Spring Data Repositories, Spring’s Transaction Management, and so on) and Spring Data for Pivotal GemFire takes care of all the Pivotal GemFire plumbing required by those framework services on the your behalf.

Pushing configuration from the client to the servers in the cluster and having the cluster remember it is made possible in part by the use of Pivotal GemFire’s Cluster Configuration service. Pivotal GemFire’s Cluster Configuration service is also the same service used by Gfsh to record schema-related changes (for example, gfsh> create region --name=Example --type=PARTITION) issued by the user to the cluster from the shell.

Of course, since the cluster may “remember” the prior configuration pushed by a client from a previous run, Spring Data for Pivotal GemFire is careful not to stomp on any existing Regions and indexes already defined in the servers. This is especially important, for instance, when Regions already contain data.

Consider the power expressed in the following configuration:

You instantly get a Spring Boot application with a Pivotal GemFire ClientCache instance, Spring Data Repositories, Spring’s Cache Abstraction with Pivotal GemFire as the caching provider (where Regions and indexes are not only created on the client but pushed to the servers in the cluster).

From there, you only need to do the following:

Nothing in this case pertains to the infrastructure and plumbing required in the application’s back-end services (such as Pivotal GemFire). Database users have similar features. Now Spring and Pivotal GemFire developers can, too.

When combined with the following Spring Data for Pivotal GemFire annotations, this application really starts to take flight, with very little effort:

See the @EnableClusterConfiguration annotation Javadoc for more details.

Without a doubt, application security is extremely important, and Spring Data for Pivotal GemFire provides comprehensive support for securing both Pivotal GemFire clients and servers.

Recently, Pivotal GemFire introduced a new Integrated Security framework (replacing its old authentication and authorization security model) for handling authentication and authorization. One of the main features and benefits of this new security framework is that it integrates with Apache Shiro and can therefore delegate both authentication and authorization requests to Apache Shiro to enforce security.

The remainder of this section demonstrates how Spring Data for Pivotal GemFire can simplify Pivotal GemFire’s security story even further.

The following tips can help you get the most out of using the new annotation-based configuration model:

As we learned in the previous sections, Spring Data for Pivotal GemFire’s new annotation-based configuration model provides a tremendous amount of power. Hopefully, it lives up to its goal of making it easier for you to get started quickly and easily when using Pivotal GemFire with Spring.

Keep in mind that, when you use the new annotations, you can still use Java configuration or XML configuration. You can even combine all three approaches by using Spring’s @Import and @ImportResource annotations on a Spring @Configuration or @SpringBootApplication class. The moment you explicitly provide a bean definition that would otherwise be provided by Spring Data for Pivotal GemFire by using an annotation, the annotation-based configuration backs away.

The annotations were not meant to handle every situation. The annotations were meant to help you get up and running as quickly and as easily as possible, especially during development.

We hope you will enjoy these new capabilities!

As with many other high-level abstractions provided by Spring, Spring Data for Pivotal GemFire provides a template to simplify Pivotal GemFire data access operations. The class provides several methods containing common Region operations, but also provides the capability to execute code against native Pivotal GemFire APIs without having to deal with Pivotal GemFire checked exceptions by using a GemfireCallback.

The template class requires a Pivotal GemFire Region, and once configured, is thread-safe and is reusable across multiple application classes:

Once the template is configured, a developer can use it alongside GemfireCallback to work directly with the Pivotal GemFire Region without having to deal with checked exceptions, threading or resource management concerns:

For accessing the full power of the Pivotal GemFire query language, a developer can use the find and findUnique methods, which, compared to the query method, can execute queries across multiple Regions, execute projections, and the like.

The find method should be used when the query selects multiple items (through SelectResults) and the latter, findUnique, as the name suggests, when only one object is returned.

Using a new data access technology requires not only accommodating a new API but also handling exceptions specific to that technology.

To accommodate the exception handling case, the Spring Framework provides a technology agnostic and consistent exception hierarchy that abstracts the application from proprietary, and usually "checked", exceptions to a set of focused runtime exceptions.

As mentioned in Spring Framework’s documentation, Exception translation can be applied transparently to your Data Access Objects (DAO) through the use of the @Repository annotation and AOP by defining a PersistenceExceptionTranslationPostProcessor bean. The same exception translation functionality is enabled when using Pivotal GemFire as long as the CacheFactoryBean is declared, e.g. using either a <gfe:cache/> or <gfe:client-cache> declaration, which acts as an exception translator and is automatically detected by the Spring infrastructure and used accordingly.

One of the most popular features of the Spring Framework is Transaction Management.

If you are not familiar with Spring’s transaction abstraction then we strongly recommend reading about Spring’s Transaction Management infrastructure as it offers a consistent programming model that works transparently across multiple APIs and can be configured either programmatically or declaratively (the most popular choice).

For Pivotal GemFire, Spring Data for Pivotal GemFire provides a dedicated, per-cache, PlatformTransactionManager that, once declared, allows Region operations to be executed atomically through Spring:

Currently, Pivotal GemFire supports optimistic transactions with read committed isolation. Furthermore, to guarantee this isolation, developers should avoid making in-place changes that manually modify values present in the cache. To prevent this from happening, the transaction manager configures the cache to use copy on read semantics by default, meaning a clone of the actual value is created each time a read is performed. This behavior can be disabled if needed through the copyOnRead property.

For more information on the semantics and behavior of the underlying Geode transaction manager, please refer to the Geode CacheTransactionManager Javadoc as well as the documentation.

It is also possible for Pivotal GemFire to participate in a Global, JTA based transaction, such as a transaction managed by an Java EE Application Server (e.g. WebSphere Application Server, a.k.a. WAS) using Container Managed Transactions (CMT) along with other JTA resources.

However, unlike many other JTA "compliant" resources (e.g. JMS Message Brokers like ActiveMQ), Pivotal GemFire is not an XA compliant resource. Therefore, Pivotal GemFire must be positioned as the "Last Resource" in a JTA transaction (prepare phase) since it does not implement the 2-phase commit protocol, or rather does not handle distributed transactions.

Many managed environments with CMT maintain support for "Last Resource", non-XA compliant resources in JTA transactions though it is not actually required in the JTA spec. More information on what a non-XA compliant, "Last Resource" means can be found in Red Hat’s documentation. In fact, Red Hat’s JBoss project, Narayana is one such LGPL Open Source implementation. Narayana refers to this as "Last Resource Commit Optimization" (LRCO). More details can be found here.

However, whether you are using Pivotal GemFire in a standalone environment with an Open Source JTA Transaction Management implementation that supports "Last Resource", or a managed environment (e.g. Java EE AS such as WAS), Spring Data Geode has you covered.

There are a series of steps you must complete to properly use Pivotal GemFire as a "Last Resource" in a JTA transaction involving more than 1 transactional resource. Additionally, there can only be 1 non-XA compliant resource (e.g. Pivotal GemFire) in such an arrangement.

1) First, you must complete Steps 1-4 in Pivotal GemFire’s documentation here.

2) Referring to Step 5 in Pivotal GemFire’s documentation, Spring Data for Pivotal GemFire’s Annotation support will attempt to set the GemFireCache, copyOnRead property for you when using the @EnableGemFireAsLastResource annotation.

However, if SDG’s auto-configuration is unsuccessful then you must explicitly set the copy-on-read attribute on the <gfe:cache> or <gfe:client-cache> element in XML or the copyOnRead property of the SDG CacheFactoryBean class in JavaConfig to true. For example…​

Peer Cache XML:

Peer Cache JavaConfig:

Client Cache XML:

Client Cache JavaConfig:

3) At this point, you skip Steps 6-8 in Pivotal GemFire’s documentation and let Spring Data Geode work its magic. All you need do is annotate your Spring @Configuration class with Spring Data for Pivotal GemFire’s new @EnableGemFireAsLastResource annotation and a combination of Spring’s Transaction Management infrastructure and Spring Data for Pivotal GemFire’s @EnableGemFireAsLastResource configuration does the trick.

The configuration looks like this…​

The only requirements are…​

3.1) The @EnableGemFireAsLastResource annotation must be declared on the same Spring @Configuration class where Spring’s @EnableTransactionManagement annotation is also specified.

3.2) The order attribute of the @EnableTransactionManagement annotation must be explicitly set to an integer value that is not Integer.MAX_VALUE or Integer.MIN_VALUE (defaults to Integer.MAX_VALUE).

Of course, hopefully you are aware that you also need to configure Spring’s JtaTransactionManager when using JTA Transactions like so..

For more details on using Spring’s Transaction Management with JTA, see here.

Effectively, Spring Data for Pivotal GemFire’s @EnableGemFireAsLastResource annotation imports configuration containing 2 Aspect bean definitions that handles the Pivotal GemFire o.a.g.ra.GFConnectionFactory.getConnection() and o.a.g.ra.GFConnection.close() operations at the appropriate points during the transactional operation.

Specifically, the correct sequence of events are…​

This is consistent with how you, as the application developer, would code this manually if you had to use the JTA API + Pivotal GemFire API yourself, as shown in the Pivotal GemFire example.

Thankfully, Spring does the heavy lifting for you and all you need do after applying the appropriate configuration (shown above) is…​

#1 & #4 above are appropriately handled for you by Spring’s JTA based PlatformTransactionManager once the @Transactional boundary is entered by your application (i.e. when the MyTransactionSerivce.someTransactionalServiceMethod() is called).

#2 & #3 are handled by Spring Data for Pivotal GemFire’s new Aspects enabled with the @EnableGemFireAsLastResource annotation.

#3 of course is the responsibility of your application.

Indeed, with the appropriate logging configured, you will see the correct sequence of events…​

For more details on using Pivotal GemFire in JTA transactions, see here.

For more details on configuring Pivotal GemFire as a "Last Resource", see here.

A powerful functionality offered by Pivotal GemFire is Continuous Query (or CQ).

In short, CQ allows a developer to create and register an OQL query, and then automatically be notified when new data that gets added to Pivotal GemFire matches the query predicate. Spring Data for Pivotal GemFire provides dedicated support for CQs through the org.springframework.data.gemfire.listener package and its listener container; very similar in functionality and naming to the JMS integration in the Spring Framework; in fact, users familiar with the JMS support in Spring, should feel right at home.

Basically Spring Data for Pivotal GemFire allows methods on POJOs to become end-points for CQ. Simply define the query and indicate the method that should be called to be notified when there is a match. Spring Data for Pivotal GemFire takes care of the rest. This is very similar to Java EE’s message-driven bean style, but without any requirement for base class or interface implementations, based on Pivotal GemFire.

Pivotal GemFire XML configuration (usually referred to as cache.xml) allows user objects to be declared as part of the configuration. Usually these objects are CacheLoaders or other pluggable callback components supported by Pivotal GemFire. Using native Pivotal GemFire configuration, each user type declared through XML must implement the Declarable interface, which allows arbitrary parameters to be passed to the declared class through a Properties instance.

In this section, we describe how you can configure these pluggable components when defined in cache.xml using Spring while keeping your Cache/Region configuration defined in cache.xml. This allows your pluggable components to focus on the application logic and not the location or creation of DataSources or other collaborators.

However, if you are starting a green field project, it is recommended that you configure Cache, Region, and other pluggable Pivotal GemFire components directly in Spring. This avoids inheriting from the Declarable interface or the base class presented in this section.

See the following sidebar for more information on this approach.

As an example of configuring a Declarable component using Spring, consider the following declaration (taken from the Declarable Javadoc):

To simplify the task of parsing, converting the parameters and initializing the object, Spring Data for Pivotal GemFire offers a base class (WiringDeclarableSupport) that allows Pivotal GemFire user objects to be wired through a template bean definition or, in case that is missing, perform auto-wiring through the Spring IoC container. To take advantage of this feature, the user objects need to extend WiringDeclarableSupport, which automatically locates the declaring BeanFactory and performs wiring as part of the initialization process.

Spring Data for Pivotal GemFire provides an implementation of the Spring Cache Abstraction to position Pivotal GemFire as a caching provider in Spring’s caching infrastructure.

To use Pivotal GemFire as a backing implementation, a "caching provider" in Spring’s Cache Abstraction, simply add GemfireCacheManager to your configuration:

When the GemfireCacheManager (Singleton) bean instance is declared and declarative caching is enabled (either in XML with <cache:annotation-driven/> or in JavaConfig with Spring’s @EnableCaching annotation), the Spring caching annotations (e.g. @Cacheable) identify the "caches" that will cache data in-memory using Pivotal GemFire Regions.

These caches (i.e. Regions) must exist before the caching annotations that use them otherwise an error will occur.

By way of example, suppose you have a Customer Service application with a CustomerService application component that performs caching…​

Then you will need the following config.

XML:

JavaConfig:

Of course, you are free to choose whatever Region type you like (e.g. REPLICATE, PARTITION, LOCAL, etc).

For more details on Spring’s Cache Abstraction, again, please refer to the documentation.

It is fairly common for serialized objects to have transient data. Transient data is often dependent on the system or environment where it lives at a certain point in time. For instance, a DataSource is environment specific. Serializing such information is useless and potentially even dangerous, since it is local to a certain VM or machine. For such cases, Spring Data for Pivotal GemFire offers a special Instantiator that performs wiring for each new instance created by Pivotal GemFire during deserialization.

Through such a mechanism, you can rely on the Spring container to inject and manage certain dependencies, making it easy to split transient from persistent data and have rich domain objects in a transparent manner.

Spring users might find this approach similar to that of @Configurable). The WiringInstantiator works similarly to WiringDeclarableSupport, trying to first locate a bean definition as a wiring template and otherwise falling back to auto-wiring.

See the previous section (Wiring Declarable Components) for more details on wiring functionality.

To use the SDG Instantiator, declare it as a bean, as the following example shows:

During the Spring container startup, once it has been initialized, the Instantiator, by default, registers itself with the Pivotal GemFire serialization system and performs wiring on all instances of SomeDataSerializableClass created by Pivotal GemFire during deserialization.

For data intensive applications, a large number of instances might be created on each machine as data flows in. Pivotal GemFire uses reflection to create new types, but, for some scenarios, this might prove to be expensive. As always, it is good to perform profiling to quantify whether this is the case or not. For such cases, Spring Data for Pivotal GemFire allows the automatic generation of Instatiator classes, which instantiate a new type (using the default constructor) without the use of reflection. The following example shows how to create an instantiator:

The preceding definition automatically generates two Instantiators for two classes (CustomTypeA and CustomTypeB) and registers them with Pivotal GemFire under user ID 1025 and 1026. The two Instantiators avoid the use of reflection and create the instances directly through Java code.

Spring Data for Pivotal GemFire provides support to map entities that are stored in a Region. The mapping metadata is defined by using annotations on application domain classes, as the following example shows:

The @Region annotation can be used to customize the Region in which an instance of the Person class is stored. The @Id annotation can be used to annotate the property that should be used as the cache Region key, identifying the Region entry. The @PersistenceConstructor annotation helps to disambiguate multiple potentially available constructors, taking parameters and explicitly marking the constructor annotated as the constructor to be used to construct entities. In an application domain class with no or only a single constructor, you can omit the annotation.

In addition to storing entities in top-level Regions, entities can be stored in Sub-Regions as well, as the following example shows:

Be sure to use the full path of the Pivotal GemFire Region, as defined with the Spring Data for Pivotal GemFire XML namespace by using the id or name attributes of the <*-region> element.

As an alternative to specifying the Region in which the entity is stored by using the @Region annotation on the entity class, you can also specify the @Region annotation on the entity’s Repository interface. See Spring Data for Pivotal GemFire Repositories for more details.

However, suppose you want to store a Person record in multiple Pivotal GemFire Regions (for example, People and Customers). Then you can define your corresponding Repository interface extensions as follows:

Then, using each Repository individually, you can store the entity in multiple Pivotal GemFire Regions, as the following example shows:

You can even wrap the update service method in a Spring managed transaction, either as a local cache transaction or a global transaction.

Spring Data for Pivotal GemFire provides a custom PdxSerializer implementation that uses the mapping information to customize entity serialization.

It also lets you customize entity instantiation by using the Spring Data EntityInstantiator abstraction. By default, the serializer uses a ReflectionEntityInstantiator that uses the persistence constructor of the mapped entity (the default constructor, a singly declared constructor, or a constructor explicitly annotated with @PersistenceConstructor).

To provide arguments for constructor parameters, the serializer reads fields with the named constructor parameter, explicitly specified using Spring’s @Value annotation, from the supplied PdxReader, as shown in the following example:

An entity class annotated in this way has the thing field read from the PdxReader and passed as the value for the constructor parameter, firstname. The value for lastName is a Spring bean with the name “bean”.

In addition to the custom instantiation logic and strategy provided by EntityInstantiators, the MappingPdxSerializer also provides capabilities well beyond Pivotal GemFire’s own ReflectionBasedAutoSerializer.

While Pivotal GemFire’s ReflectionBasedAutoSerializer conveniently uses Java reflection to populate entities and uses regular expressions to identify types that should be handled (serialized and deserialized) by the ReflectionBasedAutoSerializer, it cannot, unlike MappingPdxSerializer, perform the following:

We now explore each feature of the MappingPdxSerializer in a bit more detail.

To bootstrap Spring Data Repositories, use the <repositories/> element from the Spring Data for Pivotal GemFire Data namespace, as the following example shows:

The preceding configuration snippet looks for interfaces below the configured base package and creates Repository instances for those interfaces backed by a SimpleGemFireRepository.

Alternatively, many developers prefer to use Spring’s Java-based container configuration.

Using this approach, you can bootstrap Spring Data Repositories by using the SDG @EnableGemfireRepositories annotation, as the following example shows:

Rather than use the basePackages attribute, you may prefer to use the type-safe basePackageClasses attribute instead. The basePackageClasses lets you specify the package that contains all your application Repository classes by specifying only one of your application Repository interface types. Consider creating a special no-op marker class or interface in each package that serves no purpose other than to identify the location of application Repositories referenced by this attribute.

In addition to the basePackages and basePackageClasses attributes, like Spring’s @ComponentScan annotation, the @EnableGemfireRepositories annotation provides include and exclude filters, based on Spring’s ComponentScan.Filter type. You can use the filterType attribute to filter by different aspects, such as whether an application Repository type is annotated with a particular annotation or extends a particular class type and so on. See the FilterType Javadoc for more details.

The @EnableGemfireRepositories annotation also lets you specify the location of named OQL queries, which reside in a Java Properties file, by using the namedQueriesLocation attribute. The property name must match the name of a Repository query method and the property value is the OQL query you want executed when the Repository query method is called.

The repositoryImplementationPostfix attribute can be set to an alternate value (defaults to Impl) if your application requires one or more custom repository implementations. This feature is commonly used to extend the Spring Data Repository infrastructure to implement a feature not provided by the data store (for example, SDG).

One example of where custom repository implementations are needed with Pivotal GemFire is when performing joins. Joins are not supported by SDG Repositories. With a Pivotal GemFire PARTITION Region, the join must be performed on collocated PARTITION Regions, since Pivotal GemFire does not support “distributed” joins. In addition, the Equi-Join OQL Query must be performed inside a Pivotal GemFire Function. See here for more details on Pivotal GemFire Equi-Join Queries.

Many other aspects of the SDG’s Repository infrastructure extension may be customized as well. See the @EnableGemfireRepositories Javadoc for more details on all configuration settings.

Spring Data for Pivotal GemFire Repositories enable the definition of query methods to easily execute Pivotal GemFire OQL queries against the Region the managed entity maps to, as the following example shows:

The first query method listed in the preceding example causes the following OQL query to be derived: SELECT x FROM /People x WHERE x.emailAddress = $1. The second query method works the same way except it returns all entities found, whereas the first query method expects a single result to be found.

If the supported keywords are not sufficient to declare and express your OQL query, or the method name becomes too verbose, then you can annotate the query methods with @Query as shown on the third and fourth methods.

The following table gives brief samples of the supported keywords that you can use in query methods:

Many query languages, such as Pivotal GemFire’s OQL (Object Query Language), have extensions that are not directly supported by Spring Data Commons' Repository infrastructure.

One of Spring Data Commons' Repository infrastructure goals is to function as the lowest common denominator to maintain support for and portability across the widest array of data stores available and in use for application development today. Technically, this means developers can access multiple different data stores supported by Spring Data Commons within their applications by reusing their existing application-specific Repository interfaces — a convenient and powerful abstraction.

To support Pivotal GemFire’s OQL Query language extensions and preserve portability across different data stores, Spring Data for Pivotal GemFire adds support for OQL Query extensions by using Java annotations. These annotations are ignored by other Spring Data Repository implementations (such as Spring Data JPA or Spring Data Redis) that do not have similar query language features.

For instance, many data stores most likely do not implement Pivotal GemFire’s OQL IMPORT keyword. Implementing IMPORT as an annotation (that is, @Import) rather than as part of the query method signature (specifically, the method 'name') does not interfere with the parsing infrastructure when evaluating the query method name to construct another data store language appropriate query.

Currently, the set of Pivotal GemFire OQL Query language extensions that are supported by Spring Data for Pivotal GemFire include:

As an example, suppose you have a Customers application domain class and corresponding Pivotal GemFire Region along with a CustomerRepository and a query method to lookup Customers by last name, as follows:

The preceding example results in the following OQL Query:

<TRACE> <HINT 'LastNameIdx'> IMPORT org.example.app.domain.Customer; SELECT * FROM /Customers x WHERE x.lastName = $1 LIMIT 10

Spring Data for Pivotal GemFire’s Repository extension is careful not to create conflicting declarations when the OQL annotation extensions are used in combination with the @Query annotation.

As another example, suppose you have a raw @Query annotated query method defined in your CustomerRepository, as follows:

The preceding query method results in the following OQL query:

IMPORT org.example.app.domain.Customer; <TRACE> <HINT 'ReputationIdx'> SELECT DISTINCT * FROM /Customers x WHERE x.reputation > $1 ORDER BY c.reputation DESC LIMIT 5

The @Limit(10) annotation does not override the LIMIT explicitly defined in the raw query. Also, the @Hint("CustomerIdx") annotation does not override the HINT explicitly defined in the raw query. Finally, the @Trace annotation is redundant and has no additional effect.

Thanks to using the Spring Data Repository abstraction, the query method convention for defining data store specific queries (e.g. OQL) is easy and convenient. However, it is sometimes desirable to still want to inspect or even possibly modify the query generated from the Repository query method.

Since 2.0.x, Spring Data for Pivotal GemFire includes the o.s.d.gemfire.repository.query.QueryPostProcessor functional interface. The interface is loosely defined as follows:

There are additional default methods provided that let you compose instances of QueryPostProcessor similar to how java.util.function.Function.andThen(:Function) and java.util.function.Function.compose(:Function) work.

Additionally, the QueryPostProcessor interface implements the org.springframework.core.Ordered interface, which is useful when multiple QueryPostProcessors are declared and registered in the Spring container and used to create a pipeline of processing for a group of generated query method queries.

Finally, the QueryPostProcessor accepts type arguments corresponding to the type parameters, T and QUERY, respectively. Type T extends the Spring Data Commons marker interface, org.springframework.data.repository.Repository. We discuss this further later in this section. All QUERY type parameter arguments in Spring Data for Pivotal GemFire’s case are of type java.lang.String.

You can implement this interface to receive a callback with the query that was generated from the application Repository interface method when the method is called.

For example, you might want to log all queries from all application Repository interface definitions. You could do so by using the following QueryPostProcessor implementation:

The LoggingQueryPostProcessor was typed to the Spring Data org.springframework.data.repository.Repository marker interface, and, therefore, logs all application Repository interface query method generated queries.

You could limit the scope of this logging to queries only from certain types of application Repository interfaces, such as, say, a CustomerRepository, as the following example shows:

Then you could have typed the LoggingQueryPostProcessor specifically to the CustomerRepository, as follows:

As a result, only queries defined in the CustomerRepository interface, such as findByAccountNumber, are logged.

You might want to create a QueryPostProcessor for a specific query defined by a Repository query method. For example, suppose you want to limit the OQL query generated from the CustomerRepository.findByLastNameLike(:String) query method to only return five results along with ordering the Customers by firstName, in ascending order . To do so, you can define a custom QueryPostProcessor, as the following example shows:

While the preceding example works, you can achieve the same effect by using the Spring Data Repository convention provided by Spring Data for Pivotal GemFire. For instance, the same query could be defined as follows:

However, if you do not have control over the application CustomerRepository interface definition, then the QueryPostProcessor (that is, OrderedLimitedCustomerByLastNameQueryPostProcessor) is convenient.

If you want to ensure that the LoggingQueryPostProcessor always comes after the other application-defined QueryPostProcessors that may have bean declared and registered in the Spring ApplicationContext, you can set the order property by overriding the o.s.core.Ordered.getOrder() method, as the following example shows:

This ensures that you always see the effects of the post processing applied by other QueryPostProcessors before the LoggingQueryPostProcessor logs the query.

You can define as many QueryPostProcessors in the Spring ApplicationContext as you like and apply them in any order, to all or specific application Repository interfaces, and be as granular as you like by using the provided arguments to the postProcess(..) method callback.

There are two separate concerns to address: implementation and execution.

The first is Function implementation (server-side), which must interact with the FunctionContext to access the invocation arguments, ResultsSender to send results, and other execution context information. The Function implementation typically accesses the cache and Regions and is registered with the FunctionService under a unique ID.

A cache client application invoking a Function does not depend on the implementation. To invoke a Function, the application instantiates an Execution providing the Function ID, invocation arguments, and the Function target, which defines its scope: Region, server, servers, member, or members. If the Function produces a result, the invoker uses a ResultCollector to aggregate and acquire the execution results. In certain cases, a custom ResultCollector implementation is required and may be registered with the Execution.

Using Pivotal GemFire APIs, the FunctionContext provides a runtime invocation context that includes the client’s calling arguments and a ResultSender implementation to send results back to the client. Additionally, if the Function is executed on a Region, the FunctionContext is actually an instance of RegionFunctionContext, which provides additional information, such as the target Region on which the Function was invoked, any filter (a set of specific keys) associated with the Execution, and so on. If the Region is a PARTITION Region, the Function should use the PartitionRegionHelper to extract the local data set.

By using Spring, you can write a simple POJO and use the Spring container to bind one or more of your POJO’s public methods to a Function. The signature for a POJO method intended to be used as a Function must generally conform to the client’s execution arguments. However, in the case of a Region execution, the Region data may also be provided (presumably the data is held in the local partition if the Region is a PARTITION Region).

Additionally, the Function may require the filter that was applied, if any. This suggests that the client and server share a contract for the calling arguments but that the method signature may include additional parameters to pass values provided by the FunctionContext. One possibility is for the client and server to share a common interface, but this is not strictly required. The only constraint is that the method signature includes the same sequence of calling arguments with which the Function was invoked after the additional parameters are resolved.

For example, suppose the client provides a String and an int as the calling arguments. These are provided in the FunctionContext as an array, as the following example shows:

Object[] args = new Object[] { "test", 123 };

The Spring container should be able to bind to any method signature similar to the following (ignoring the return type for the moment):

The general rule is that once any additional arguments (that is, Region data and filter) are resolved, the remaining arguments must correspond exactly, in order and type, to the expected Function method parameters. The method’s return type must be void or a type that may be serialized (as a java.io.Serializable, DataSerializable, or PdxSerializable). The latter is also a requirement for the calling arguments.

The Region data should normally be defined as a Map, to facilitate unit testing, but may also be of type Region, if necessary. As shown in the preceding example, it is also valid to pass the FunctionContext itself or the ResultSender if you need to control over how the results are returned to the client.

A process that invokes a remote Function needs to provide the Function’s ID, calling arguments, the execution target (onRegion, onServers, onServer, onMember, or onMembers) and (optionally) a filter set. By using Spring Data for Pivotal GemFire, all you need do is define an interface supported by annotations. Spring creates a dynamic proxy for the interface, which uses the FunctionService to create an Execution, invoke the Execution, and (if necessary) coerce the results to the defined return type. This technique is similar to the way Spring Data for Pivotal GemFire’s Repository extension works. Thus, some of the configuration and concepts should be familiar.

Generally, a single interface definition maps to multiple Function executions, one corresponding to each method defined in the interface.

Using the Function execution annotated interface defined in the previous section, simply auto-wire your interface into an application bean that will invoke the Function:

Alternately, you can use a Function execution template directly. In the following example, the GemfireOnRegionFunctionTemplate creates an onRegion Function Execution:

Internally, Function Executions always return a List. executeAndExtract assumes a singleton List containing the result and attempts to coerce that value into the requested type. There is also an execute method that returns the List as is. The first parameter is the Function ID. The filter argument is optional. The remaining arguments are a variable argument List.

When using Spring Data for Pivotal GemFire’s Function annotation support combined with Pivotal GemFire’s PDX Serialization, there are a few logistical things to keep in mind.

As explained earlier in this section, and by way of example, you should typically define Pivotal GemFire Functions by using POJO classes annotated with Spring Data for Pivotal GemFire Function annotations, as follows:

Your Order class and OrderSource enum might be defined as follows:

Of course, you can define a Function Execution interface to call the 'process' Pivotal GemFire server Function, as follows:

Clearly, this process(..) Order Function is being called from the client-side with a ClientCache instance (that is <gfe:client-cache/>). This implies that the Function arguments must also be serializable. The same is true when invoking peer-to-peer member Functions (such as @OnMember(s)) between peers in the cluster. Any form of distribution requires the data transmitted between client and server (or peers) to be serialized.

Now, if you have configured Pivotal GemFire to use PDX for serialization (instead of Java serialization, for instance) you can also set the pdx-read-serialized attribute to true in your configuration of the Pivotal GemFire server(s), as follows:

Alternatively, you can set the pdx-read-serialized attribute to true for a Pivotal GemFire cache client application, as follows:

Doing so causes all values read from the cache (that is, Regions) as well as information passed between client and servers (or peers) to remain in serialized form, including, but not limited to, Function arguments.

Pivotal GemFire serializes only application domain object types that you have specifically configured (registered) either by using Pivotal GemFire’s ReflectionBasedAutoSerializer, or specifically (and recommended) by using a “custom” Pivotal GemFire PdxSerializer. If you use Spring Data for Pivotal GemFire’s Repository extension, you might even want to consider using Spring Data for Pivotal GemFire’s MappingPdxSerializer, which uses an entity’s mapping metadata to determine data from the application domain object that is serialized to the PDX instance.

What is less than apparent, though, is that Pivotal GemFire automatically handles Java Enum types regardless of whether they are explicitly configured (that is, registered with a ReflectionBasedAutoSerializer, using a regex pattern and the classes parameter or are handled by a “custom” Pivotal GemFire PdxSerializer), despite the fact that Java enumerations implement java.io.Serializable.

So, when you set pdx-read-serialized to true on Pivotal GemFire servers where the Pivotal GemFire Functions (including Spring Data for Pivotal GemFire Function-annotated POJO classes) are registered, then you may encounter surprising behavior when invoking the Function Execution.

You might pass the following arguments when invoking the Function:

However, the Pivotal GemFire Function on the server gets the following:

The Order and OrderSource have been passed to the Function as PDX instances. Again, this all happens because pdx-read-serialized is set to true, which may be necessary in cases where the Pivotal GemFire servers interact with multiple different clients (for example, a combination of Java clients and native clients, such as C/C++, C#, and others).

This flies in the face of Spring Data for Pivotal GemFire’s strongly-typed Function-annotated POJO class method signatures, where you would reasonably expect application domain object types instead, not PDX serialized instances.

Consequently, Spring Data for Pivotal GemFire includes enhanced Function support to automatically convert PDX typed method arguments to the desired application domain object types defined by the Function method’s signature (parameter types).

However, this also requires you to explicitly register a Pivotal GemFire PdxSerializer on Pivotal GemFire servers where Spring Data for Pivotal GemFire Function-annotated POJOs are registered and used, as the following example shows:

Alternatively, you can use Pivotal GemFire’s ReflectionBasedAutoSerializer for convenience. Of course, we recommend that, where possible, you use a custom PdxSerializer to maintain finer-grained control over your serialization strategy.

Finally, Spring Data for Pivotal GemFire is careful not to convert your Function arguments if you treat your Function arguments generically or as one of Pivotal GemFire’s PDX types, as follows:

Spring Data for Pivotal GemFire converts PDX typed data to the corresponding application domain types if and only if the corresponding application domain types are on the classpath and the Function-annotated POJO method expects it.

For a good example of custom, composed application-specific Pivotal GemFire PdxSerializers as well as appropriate POJO Function parameter type handling based on the method signatures, see Spring Data for Pivotal GemFire’s ClientCacheFunctionExecutionWithPdxIntegrationTest class.

Spring Data for Pivotal GemFire provides two primary templates for Lucene data access operations, depending on how low of a level your application is prepared to deal with.

The LuceneOperations interface defines query operations by using Pivotal GemFire Lucene types, which are defined in the following interface definition:

The operations in the LuceneOperations interface match the operations provided by the Pivotal GemFire’s LuceneQuery interface. However, SDG has the added value of translating proprietary Pivotal GemFire or Apache Lucene Exceptions into Spring’s highly consistent and expressive DAO exception hierarchy, particularly as many modern data access operations involve more than one store or repository.

Additionally, SDG’s LuceneOperations interface can shield your application from interface-breaking changes introduced by the underlying Pivotal GemFire or Apache Lucene APIs when they occur.

However, it would be sad to offer a Lucene Data Access Object (DAO) that only uses Pivotal GemFire and Apache Lucene data types (such as Pivotal GemFire’s LuceneResultStruct). Therefore, SDG gives you the ProjectingLuceneOperations interface to remedy these important application concerns. The following listing shows the ProjectingLuceneOperations interface definition:

The ProjectingLuceneOperations interface primarily uses application domain object types that let you work with your application data. The query method variants accept a projection type, and the template applies the query results to instances of the given projection type by using the Spring Data Commons Projection infrastructure.

Additionally, the template wraps the paged Lucene query results in an instance of the Spring Data Commons Page abstraction. The same projection logic can still be applied to the results in the page and are lazily projected as each page in the collection is accessed.

By way of example, suppose you have a class representing a Person, as follows:

Additionally, you might have a single interface to represent people as Customers, depending on your application view, as follows:

If I define the following LuceneIndex…​

Then you could query for people as Person objects, as follows:

Alternatively, you could query for a Page of type Customer, as follows:

The Page can then be used to fetch individual pages of the results, as follows:

Conveniently, the Spring Data Commons Page interface also implements java.lang.Iterable<T>, making it easy to iterate over the contents.

The only restriction to the Spring Data Commons Projection infrastructure is that the projection type must be an interface. However, it is possible to extend the provided SDC Projection infrastructure and provide a custom ProjectionFactory that uses CGLIB to generate proxy classes as the projected entity.

You can use setProjectionFactory(:ProjectionFactory) to set a custom ProjectionFactory on a Lucene template.

Finally, Spring Data for Pivotal GemFire provides annotation configuration support for LuceneIndexes.

Eventually, the SDG Lucene support will finds its way into the Repository infrastructure extension for Pivotal GemFire so that Lucene queries can be expressed as methods on an application Repository interface, in much the same way as the OQL support works today.

However, in the meantime, if you want to conveniently express LuceneIndexes, you can do so directly on your application domain objects, as the following example shows:

To enable this feature, you must use SDG’s annotation configuration support specifically with the @EnableEntityDefineRegions and @EnableIndexing annotations, as follows:

Given our earlier definition of the Person class, the SDG annotation configuration support finds the Person entity class definition and determines that people are stored in a PARTITION Region called “People” and that the Person has an OQL Index on birthDate along with a LuceneIndex on lastName.

In order to bootstrap a Spring ApplicationContext in Pivotal GemFire when starting a Pivotal GemFire server using Gfsh, you must use Pivotal GemFire’s initalizer capability. An initializer block can declare a application callback that is launched after the cache is initialized by Pivotal GemFire.

An initializer is declared within an initializer element by using a minimal snippet of Pivotal GemFire’s native cache.xml. To bootstrap the Spring ApplicationContext, a cache.xml file is required, in much the same way as a minimal snippet of Spring XML config is needed to bootstrap a Spring ApplicationContext configured with component scanning (for example <context:component-scan base-packages="…​"/>).

Fortunately, such an initializer is already conveniently provided by the framework: the SpringContextBootstrappingInitializer.

The following example shows a typical, yet minimal, configuration for this class inside Pivotal GemFire’s cache.xml file:

The SpringContextBootstrappingInitializer class follows conventions similar to Spring’s ContextLoaderListener class, which is used to bootstrap a Spring ApplicationContext inside a web application, where ApplicationContext configuration files are specified with the contextConfigLocations Servlet context parameter.

In addition, the SpringContextBootstrappingInitializer class can also be used with a basePackages parameter to specify a comma-separated list of base packages that contain appropriately annotated application components. The Spring container searches these components to find and create Spring beans and other application components in the classpath, as the following example shows:

Then, with a properly configured and constructed CLASSPATH and cache.xml file (shown earlier) specified as a command-line option when starting a Pivotal GemFire server in Gfsh, the command-line would be as follows:

The application-context.xml can be any valid Spring configuration metadata, including all of the SDG XML namespace elements. The only limitation with this approach is that a Pivotal GemFire cache cannot be configured by using the SDG XML namespace. In other words, none of the <gfe:cache/> element attributes (such as cache-xml-location, properties-ref, critical-heap-percentage, pdx-serializer-ref, lock-lease, and others) can be specified. If used, these attributes are ignored.

The reason for this is that Pivotal GemFire itself has already created and initialized the cache before the initializer gets invoked. As a result, the cache already exists and, since it is a “singleton”, it cannot be re-initialized or have any of its configuration augmented.

Spring Data for Pivotal GemFire already provides support for auto-wiring Pivotal GemFire components (such as CacheListeners, CacheLoaders, CacheWriters and so on) that are declared and created by Pivotal GemFire in cache.xml by using SDG’s WiringDeclarableSupport class, as described in Configuration using auto-wiring and annotations. However, this works only when Spring is the one doing the bootstrapping (that is, when Spring bootstraps Pivotal GemFire).

When your Spring ApplicationContext is bootstrapped by Pivotal GemFire, these Pivotal GemFire application components go unnoticed, because the Spring ApplicationContext does not exist yet. The Spring ApplicationContext does not get created until Pivotal GemFire calls the initializer block, which only occurs after all the other Pivotal GemFire components (cache, Regions, and others) have already been created and initialized.

To solve this problem, a new LazyWiringDeclarableSupport class was introduced. This new class is aware of the Spring ApplicationContext. The intention behind this abstract base class is that any implementing class registers itself to be configured by the Spring container that is eventually created by Pivotal GemFire once the initializer is called. In essence, this gives your Pivotal GemFire application components a chance to be configured and auto-wired with Spring beans defined in the Spring container.

In order for your Pivotal GemFire application components to be auto-wired by the Spring container, you should create an application class that extends the LazyWiringDeclarableSupport and annotate any class member that needs to be provided as a Spring bean dependency, similar to the following example:

As implied in the CacheLoader example above, you might necessarily (though rarely) have defined both a Region and a CacheListener component in Pivotal GemFire cache.xml. The CacheLoader may need access to an application Repository (or perhaps a JDBC DataSource defined in the Spring ApplicationContext) for loading Users into a Pivotal GemFire REPLICATE Region on startup.

CAUTION

The “Hello World” sample application demonstrates the core functionality of the Spring Data for Pivotal GemFire project. It bootstraps Pivotal GemFire, configures it, executes arbitrary commands against the cache, and shuts it down when the application exits. Multiple instances of the application can be started at the same time and work together, sharing data without any user intervention.