Look-Aside Caching with Spring

Caching is an effective software design pattern for reducing the resource consumption used by your application as well as improving efficiency by increasing throughput and reducing latency.

The fundamental premise of caching is, when given the same arguments, if a service call yields the same results every time, then it is a good candidate for caching.

Indeed, if I am searching for a customer record by account number and the search always yields the same customer for a given account number, then adding caching to the search operation will improve the overall user experience. After all, the account number may be a form of customer identity. We can save compute resources by caching the customer’s information, which is especially useful if the customer’s information is used in multiple workflows of the application during the interactions with the customer.

While there are different patterns of caching, the Look-Aside Caching pattern is the most frequently used.

Look-Aside Caching is a pattern of caching where the input of a cacheable operation is used as the key for looking up any cached results from a prior invocation of the operation when given the same input. In Look-Aside Caching, the cache is consulted first, before the operation is invoked, and if a computation for the given input has already been computed and cached, then the value from the cache is returned. Otherwise, if no value has been cached for the given input, or the previous cache result expired, or was evicted, then the operation will be invoked and the result of the operation is cached using the input as the key and the result as a value.

It should be apparent that the data structure of a cache is a key/value store, or a Map. Indeed it is quite common for most cache implementations to even implement the java.util.Map interface. However, many cache implementations are quite a bit more sophisticated, providing distribution (to scale-out), replication (HA) and even persistence along with other capabilities.

For example, I may have a CustomerService class that looks up a Customer by AccountNumber:

If I have already looked up a Customer (e.g. "Jon Doe") with a given AccountNumber (e.g. "abc123"), then when the findBy(..) method is called with the same AccountNumber (i.e. "abc123") again, we would expect the same result (i.e. Customer "Jon Doe") to be returned.

The Look-Aside Caching pattern can be depicted in the following diagram:

In the diagram above, we see that the caching provider (e.g. Apache Geode) is consulted first, #2, after the client initiated the request, #1. If the result of the cacheable operation for the given input has already been computed and stored in the cache (a cache hit), then the result is simply returned, #3, and passed back to the caller, #6.

However, if the cacheable operation has never been invoked with the given input, or the previous computation of the operation for the given input expired, or was evicted, then the cacheable operation is invoked (cache miss). This cacheable operation may access some external data source to perform its computation, #3 (red). After the operation completes, it returns the result, but not before the caching infrastructure stores the result along with the input in the cache, #4 & #5. After the result is cached, the value is returned to the caller, #3 (green). Any subsequent invocation of the cacheable operation with the same input should yield the same result as stored in the cache, providing the cache entry (input→result) has not expired or been evicted.

Spring’s Cache Abstraction is just that, a very elegant implementation of the Look-Aside Caching pattern. Details of how Spring’s Cache Abstraction works under-the-hood is beyond the scope of this document. In a nutshell, it relies on Spring AOP and is not unlike Spring’s Transaction Management.

Different caching providers have different capabilities. You should choose the caching provider that gives you what you require to handle your application needs and use cases correctly.

If used appropriately, caching can greatly improve your application’s end-user experience.

To make the effects of Spring’s Cache Abstraction using Apache Geode as the cache provider apparent in your application, we show how to enable and use caching with your application in a very small, simple example.

The example Spring Boot application implements a Counter Service, which simply maintains a collection of named counters. The application provides a REST-ful Web interface to increment a counter, get the current cached count for a named counter, and the ability to reset a named counter to 0.

Typically, caching is used to offset the costs associated with expensive operations, such as disk or network I/O. Indeed, both an operation’s throughput and latency is bound by an I/O operation since compute is many orders of magnitude faster than disk, network, etc.

While developers have been quick to throw more Threads at the problem, trying to do more work in parallel, this opens the door to a whole new set of problems (concurrency), usually at the expense of using more resources, which does not always yield the desired results.

Opportunities for caching are often overlooked, yet is very effective at minimizing the over utilization of resources by leveraging reuse. In an ever increasing Microservices based world, caching will become even more important as it serves a very important role in the applications architecture, not the least of which is, resiliency.

Of course, you still must tune your cache. Most caches keep information in memory, and since memory is finite, you must utilize strategies to manage memory effectively, such as eviction, expiration, or even Off-Heap (i.e. native memory) for JVM-based caches. For example, evicting/expiring entries based on use (Least Recently Used, or LRU) is 1 of many effective strategies.

Each caching provider’s capabilities are different in this regard. The choice should not only be based on what capabilities you need now, but capabilities (e.g. distributed compute, streaming) you may need in the future. So, choose wisely.

Now, it is time to run the example.

If you are just running in local mode (provided configuration), then start the BootGeodeLookAsideCachingApplication from your IDE, or from the command-line, as is:

Then open your Web browser and navigate to http://localhost:8080/ping:

After that, we can create and increment counters, for example:

http://localhost:8080/counter/A

1

If you constantly hit the refresh button, you will see 2, 3, 4, 5, …​ and so on. While the named counter’s (i.e. "A") new count is being cached, we are not returning the cached value.

If you navigate to:

http://localhost:8080/counter/A/cached

The count for the named counter (e.g. "A") will remain fixed on whatever the last count was (e.g. "5").

You can begin a new named counter (e.g. "B") without affecting the exiting named counter (i.e. "A"), by navigating to:

http://localhost:8080/counter/B

1

And again, after refreshing the page multiple times:

3

If you navigate to:

http://localhost:8080/counter/B/reset

0

This resets the count of counter "B" to 0. However, this does not affect the count of counter "A", which we can reevaluate by navigating to:

http://localhost:8080/counter/A/cached

5

This is an extremely simple application, but shows the effects of caching.

As you have learned, Spring makes enabling and using caching in your application really easy.

With SBDG, using either Apache Geode or Pivotal GemFire (PCC) as your caching provider in Spring’s Cache Abstraction is as easy as making sure org.springframework.geode:spring-geode-starter is on your application’s classpath. You just need to focus on areas of your application that would benefit from caching.

You now have successfully used the Look-Aside Caching pattern in your Spring Boot application.

Later we will cover more advanced forms of the Look-Aside Caching pattern (e.g. using Eviction/Expiration policies) as well as take a look at other caching patterns, like Inline Caching and Near Caching.

Back