Spring Cloud GCP Reference Documentation

João André Martins

Table of Contents

1. Introduction
2. Dependency Management
3. Spring Cloud GCP Core
3.1. Project ID
3.2. Credentials
3.3. Scopes
4. Spring Cloud GCP for Pub/Sub
4.1. Pub/Sub operations abstraction
4.1.1. Publishing to a topic
4.2. Pub/Sub management
4.2.1. Creating a topic
4.2.2. Deleting a topic
4.2.3. Listing topics
4.2.4. Creating a subscription
4.2.5. Deleting a subscription
4.2.6. Listing subscriptions
5. Spring Resources
5.1. Google Cloud Storage
6. Spring JDBC
6.1. Spring Boot Starter for Google Cloud SQL
6.1.1. JdbcTemplate creation flow
7. Spring Integration
7.1. Channel Adapters for Google Cloud Pub/Sub
7.1.1. Inbound channel adapter
7.1.2. Outbound channel adapter

1. Introduction

The Spring Cloud GCP project aims at making the Spring Framework a first-class citizen of Google Cloud Platform (GCP).

Currently, Spring Cloud GCP lets you leverage the power and simplicity of the Spring framework to:

  1. Publish and subscribe from Google Cloud Pub/Sub topics
  2. Exchange messages with Spring Integration using Google Cloud Pub/Sub on the background
  3. Write and read from Spring Resources backed up by Google Cloud Storage
  4. Configure Spring JDBC with a few properties to use Google Cloud SQL on the background

2. Dependency Management

The Spring Cloud GCP Bill of Materials (BOM) contains the versions of all the dependencies it uses.

If you’re a Maven user, adding the following to your pom.xml file will allow you to not specify any Spring Cloud GCP dependency versions. Instead, the version of the BOM you’re using determines the versions of the used dependencies.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-gcp-dependencies</artifactId>
            <version>1.0.0.BUILD-SNAPSHOT</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

In the following sections, the Maven dependency snippets will assume you are using the Spring Cloud GCP BOM and the dependency versions are omitted.

Gradle users can achieve the same kind of BOM experience using Spring’s dependency-management-plugin Gradle plugin.

3. Spring Cloud GCP Core

At the center of every Spring Cloud GCP module are the concepts of GcpProjectIdProvider and CredentialsProvider.

Spring Cloud GCP provides a Spring Boot starter to auto-configure the core components.

Maven coordinates, using Spring Cloud GCP BOM:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-core</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    compile group: 'org.springframework.cloud', name: 'spring-cloud-gcp-starter-core', version: '1.0.0.BUILD-SNAPSHOT'
}

3.1 Project ID

GcpProjectIdProvider is a functional interface that returns a GCP project ID string.

public interface GcpProjectIdProvider {
	String getProjectId();
}

The Spring Boot GCP Core starter auto-configures a GcpProjectIdProvider. If a spring.cloud.gcp.project-id property is specified, the provided GcpProjectIdProvider returns that property value.

spring.cloud.gcp.project-id=my-gcp-project-id

Otherwise, the project ID is discovered based on a set of rules:

  1. The project ID specified by the GOOGLE_CLOUD_PROJECT environment variable
  2. The Google App Engine project ID
  3. The project ID specified in the JSON credentials file pointed by the GOOGLE_APPLICATION_CREDENTIALS environment variable
  4. The Google Cloud SDK project ID
  5. The Google Compute Engine project ID, from the Google Compute Engine Metadata Server

3.2 Credentials

CredentialsProvider is a functional interface that returns the credentials to authenticate and authorize calls to Google Cloud Client Libraries.

public interface CredentialsProvider {
  Credentials getCredentials() throws IOException;
}

The Spring Boot GCP Core starter auto-configures a CredentialsProvider. It uses the spring.cloud.gcp.credentials.location property to locate the OAuth2 private key of a Google service account. Keep in mind this property is a Spring Resource, so the credentials file can be obtained from a number of different locations such as the file system, classpath, URL, etc. The next example specifies the credentials location property in the file system.

spring.cloud.gcp.credentials.location=file:/usr/local/key.json

If that property isn’t specified, the starter tries to discover credentials from a number of places:

  1. Credentials file pointed to by the GOOGLE_APPLICATION_CREDENTIALS environment variable
  2. Credentials provided by the Google Cloud SDK gcloud auth application-default login command
  3. Google App Engine built-in credentials
  4. Google Cloud Shell built-in credentials
  5. Google Compute Engine built-in credentials

3.3 Scopes

By default, the credentials provided by the Spring Cloud GCP Core Starter contain scopes for every service supported by Spring Cloud GCP.

The Spring Boot GCP Core starter allows you to configure a custom scope list for the provided credentials. To do that, specify a comma-delimited list of scopes in the spring.cloud.gcp.credentials.scopes property.

spring.cloud.gcp.credentials.scopes=https://www.googleapis.com/auth/pubsub,https://www.googleapis.com/auth/sqlservice.admin

4. Spring Cloud GCP for Pub/Sub

Spring Cloud GCP provides a Spring abstraction layer to publish to and subscribe from Google Cloud Pub/Sub topics and to create, list or delete Google Cloud Pub/Sub topics and subscriptions.

A Spring Boot starter is provided to auto-configure the various Pub/Sub components.

Maven coordinates, using Spring Cloud GCP BOM:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    compile group: 'org.springframework.cloud', name: 'spring-cloud-gcp-starter-pubsub', version: '1.0.0.BUILD-SNAPSHOT'
}

4.1 Pub/Sub operations abstraction

PubSubOperations is an abstraction that allows Spring users to use Google Cloud Pub/Sub without depending on any Google Cloud Pub/Sub API semantics. It provides the common set of operations needed to interact with Google Cloud Pub/Sub. PubSubTemplate is the default implementation of PubSubOperations and it uses the Google Cloud Java Client for Pub/Sub to interact with Google Cloud Pub/Sub.

PubSubTemplate depends on a PublisherFactory, which is a functional interface to provide a Google Cloud Java Client for Pub/Sub Publisher. The Spring Boot starter for GCP Pub/Sub auto-configures a PublisherFactory with default settings and uses the GcpProjectIdProvider and CredentialsProvider auto-configured by the Spring Boot GCP Core starter.

The PublisherFactory implementation provided by Spring Cloud GCP Pub/Sub, DefaultPublisherFactory, caches Publisher instances by topic name, in order to optimize resource utilization.

The Spring Boot starter for GCP Pub/Sub allows for the customization of the number of threads used by both the Publisher and Subscriber executors. In the example, we’re setting the number of threads used by both the Publisher and Subscriber to 2.

spring.cloud.gcp.pubsub.subscriber-executor-threads=2
spring.cloud.gcp.pubsub.publisher-executor-threads=2

The default value for both these properties is 4 threads.

4.1.1 Publishing to a topic

PubSubTemplate provides asynchronous methods to publish messages to a Google Cloud Pub/Sub topic. It supports different types of payloads, including Strings with different encodings, byte[], ByteString and PubsubMessage.

public ListenableFuture<String> publish(String topic, String payload, Map<String, String> headers)

public ListenableFuture<String> publish(String topic, String payload, Map<String, String> headers,
Charset charset)

public ListenableFuture<String> publish(String topic, byte[] payload, Map<String, String> headers)

public ListenableFuture<String> publish(String topic, ByteString payload, Map<String, String> headers)

public ListenableFuture<String> publish(String topic, PubsubMessage pubsubMessage)

Here is an example of how to publish a message to a Google Cloud Pub/Sub topic:

@Autowired
private PubSubOperations pubSubTemplate;

public void publishMessage() {
    this.pubSubTemplate.publish("topic", "your message payload", ImmutableMap.of("key1", "val1"));
}

4.2 Pub/Sub management

PubSubAdmin is the abstraction provided by Spring Cloud GCP to manage Google Cloud Pub/Sub resources. It allows for the creation, deletion and listing of topics and subscriptions.

PubSubAdmin depends on GcpProjectIdProvider and either a CredentialsProvider or a TopicAdminClient and a SubscriptionAdminClient. If given a CredentialsProvider, it creates a TopicAdminClient and a SubscriptionAdminClient with the Google Cloud Java Library for Pub/Sub defaults. The Spring Boot starter for GCP Pub/Sub auto-configures a PubSubAdmin object using the GcpProjectIdProvider and the CredentialsProvider auto-configured by the Spring Boot GCP Core starter.

4.2.1 Creating a topic

PubSubAdmin implements a method to create topics:

public Topic createTopic(String topicName)

Here is an example of how to create a Google Cloud Pub/Sub topic:

@Autowired
private PubSubAdmin admin;

public void newTopic() {
    admin.createTopic("topicName");
}

4.2.2 Deleting a topic

PubSubAdmin implements a method to delete topics:

public void deleteTopic(String topicName)

Here is an example of how to delete a Google Cloud Pub/Sub topic:

@Autowired
private PubSubAdmin admin;

public void deleteTopic() {
    admin.deleteTopic("topicName");
}

4.2.3 Listing topics

PubSubAdmin implements a method to list topics:

public List<Topic> listTopics

Here is an example of how to list every Google Cloud Pub/Sub topic name in a project:

@Autowired
private PubSubAdmin admin;

public List<String> listTopics() {
    return admin
        .listTopics()
        .stream()
        .map(Topic::getNameAsTopicName)
        .map(TopicName::getTopic)
        .collect(Collectors.toList());
}

4.2.4 Creating a subscription

PubSubAdmin implements a method to create subscriptions to existing topics:

public Subscription createSubscription(String subscriptionName, String topicName, Integer ackDeadline, String pushEndpoint)

Here is an example of how to create a Google Cloud Pub/Sub subscription:

@Autowired
private PubSubAdmin admin;

public void newSubscription() {
    admin.createSubscription("subscriptionName", "topicName", 10, “http://my.endpoint/push”);
}

Alternative methods with default settings are provided for ease of use. The default value for ackDeadline is 10 seconds. If pushEndpoint isn’t specified, the subscription uses message pulling, instead.

public Subscription createSubscription(String subscriptionName, String topicName)
public Subscription createSubscription(String subscriptionName, String topicName, Integer ackDeadline)
public Subscription createSubscription(String subscriptionName, String topicName, String pushEndpoint)

4.2.5 Deleting a subscription

PubSubAdmin implements a method to delete subscriptions:

public void deleteSubscription(String subscriptionName)

Here is an example of how to delete a Google Cloud Pub/Sub subscription:

@Autowired
private PubSubAdmin admin;

public void deleteSubscription() {
    admin.deleteSubscription("subscriptionName");
}

4.2.6 Listing subscriptions

PubSubAdmin implements a method to list subscriptions:

public List<Subscription> listSubscriptions()

Here is an example of how to list every subscription name in a project:

@Autowired
private PubSubAdmin admin;

public List<String> listSubscriptions() {
return admin
    .listSubscriptions()
    .stream()
    .map(Subscription::getNameAsSubscriptionName)
    .map(SubscriptionName::getSubscription)
    .collect(Collectors.toList());
}

5. Spring Resources

Spring Resources are an abstraction for a number of low-level resources, such as file system files, classpath files, servlet context-relative files, etc. Spring Cloud GCP adds a new resource type: a Google Cloud Storage (GCS) object.

A Spring Boot starter is provided to auto-configure the various Storage components.

Maven coordinates, using Spring Cloud GCP BOM:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-storage</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    compile group: 'org.springframework.cloud', name: 'spring-cloud-gcp-starter-storage', version: '1.0.0.BUILD-SNAPSHOT'
}

5.1 Google Cloud Storage

The Spring Resource Abstraction for Google Cloud Storage allows GCS objects to be accessed by their GCS URL:

@Value("gs://[YOUR_GCS_BUCKET]/[GCS_FILE_NAME]")
private Resource gcsResource;

This creates a Resource object that can be used to read the object, among other possible operations.

It is also possible to write to a Resource, although a cast to WriteableResource is required.

try (OutputStream os = ((WritableResource) gcsResource).getOutputStream()) {
  os.write("foo".getBytes());
}

The Spring Boot Starter for Google Cloud Storage auto-configures the Storage bean required by the spring-cloud-gcp-storage module, based on the CredentialsProvider provided by the Spring Boot GCP Core starter.

6. Spring JDBC

Spring Cloud GCP adds integrations with Spring JDBC so you can run your MySQL or PostgreSQL databases in Google Cloud SQL using Spring JDBC, or other libraries that depend on it like Spring Data JPA.

The Cloud SQL support provided by Spring Cloud GCP is in the form of a Spring Boot starter. The role of the starter is to read configuration from properties and assume default settings so that user experience connecting to MySQL is as easy as possible.

Maven coordinates, using Spring Cloud GCP BOM:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-sql</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    compile group: 'org.springframework.cloud', name: 'spring-cloud-gcp-starter-sql', version: '1.0.0.BUILD-SNAPSHOT'
}

6.1 Spring Boot Starter for Google Cloud SQL

The Spring Boot Starter for Google Cloud SQL provides an auto-configured JdbcTemplate object bean that allows for operations such as querying and modifying a database.

Note that this starter requires the Google Cloud SQL API to be enabled in in your GCP project.

@Autowired
private JdbcTemplate jdbcTemplate;

public List<Map<String, Object>> listUsers() {
    return jdbcTemplate.queryForList("SELECT * FROM user;");
}

JdbcTemplate is configured based on the following properties:

Property name

Description

Default value

Unused if specified property(ies)

spring.cloud.gcp.sql.database-type

The type of database used. Can either be mysql or postgresql.

mysql

 

spring.cloud.gcp.sql.credentials-location

File system path to the Google OAuth2 credentials private key file. Used to authenticate and authorize new connections to a Google Cloud SQL instance.

default credentials provided by the Spring GCP Core Boot starter

 

spring.cloud.gcp.sql.jdbc-url

The full JDBC URL connection string.

  

spring.cloud.gcp.sql.jdbc-driver

The JDBC driver class name.

com.mysql.jdbc.Driver for MySQL, org.postgresql.Driver for PostgreSQL or com.mysql.jdbc.GoogleDriver if MySQL is used within Google App Engine.

 

spring.cloud.gcp.sql.database-name

Name of the database to connect to.

 

spring.cloud.gcp.sql.jdbc-url

spring.cloud.gcp.sql.instance-connection-name

A string containing a Google Cloud SQL instance’s project ID, region and name, each separated by a colon. For example, my-project-id:my-region:my-instance-name.

 

spring.cloud.gcp.sql.jdbc-url

spring.cloud.gcp.sql.user-name

MySQL or PostgreSQL user name.

root

 

spring.cloud.gcp.sql.password

MySQL or PostgreSQL password.

empty string

 

spring.cloud.gcp.sql.init-fail-fast

If set to true, returns an exception if the minimum number of connections could not be created.

false

 

6.1.1 JdbcTemplate creation flow

Based on the previous properties, the Spring Boot starter for Google Cloud SQL creates a CloudSqlJdbcInfoProvider object which is used to obtain an instance’s JDBC URL and driver class name. If you provide your own CloudSqlJdbcInfoProvider bean, it is used instead and the properties related to building the JDBC URL or driver class are ignored.

It is in the CloudSqlJdbcInfoProvider creation step that the credentials factory is registered in a system property to be SqlCredentialFactory. This is especially relevant if you want to provide your own CloudSqlJdbcInfoProvider bean, in which case you should also register the credentials factory class name under the cloudSql.socketFactory.credentialFactory system property.

The CloudSqlJdbcInfoProvider bean is used by the starter to create a DataSource object. This DataSource object is picked up by the Spring JDBC module to automatically configure a JdbcTemplate object. We chose HikariCP as the default connection pool implementation, but you’re free to provide your own DataSource implementation using the provided CloudSqlJdbcInfoProvider bean to resolve an instance’s JDBC URL and driver class name.

It is in the DataSource creation step that the JDBC driver class is registered. This isn’t relevant unless you want to provide your own DataSource bean, in which case, you should load the driver class so that the user doesn’t have to.

The result of this flow is a fully configured and operational JdbcTemplate object that you can use to interact with your SQL database. You can connect to your database with as little as a database and instance names!

7. Spring Integration

Spring Cloud GCP integrates with Spring Integration by providing channel adapters that connect your Spring MessageChannels to Google Cloud Pub/Sub. This enables messaging between different processes, applications or micro-services backed up by Google Cloud Pub/Sub.

Maven coordinates, using Spring Cloud GCP BOM:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-integration-gcp</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    compile group: 'org.springframework.cloud', name: 'spring-integration-gcp', version: '1.0.0.BUILD-SNAPSHOT'
}

7.1 Channel Adapters for Google Cloud Pub/Sub

7.1.1 Inbound channel adapter

PubSubInboundChannelAdapter is the inbound channel adapter for GCP Pub/Sub that listens to a GCP Pub/Sub subscription for new messages. It converts new messages to an internal Spring Message and then sends it to the bound output channel.

To use the inbound channel adapter, a PubSubInboundChannelAdapter must be provided and configured on the user application side.

@Bean
public MessageChannel pubsubInputChannel() {
    return new PublishSubscribeChannel();
}

@Bean
public PubSubInboundChannelAdapter messageChannelAdapter(
    @Qualifier("pubsubInputChannel") MessageChannel inputChannel,
    SubscriberFactory subscriberFactory) {
    PubSubInboundChannelAdapter adapter =
        new PubSubInboundChannelAdapter(subscriberFactory, "subscriptionName");
    adapter.setOutputChannel(inputChannel);
    adapter.setAckMode(AckMode.MANUAL);

    return adapter;
}

In the example, we first specify the MessageChannel where the adapter is going to write incoming messages to. The MessageChannel implementation isn’t important here. Depending on your use case, you might want to use a MessageChannel other than PublishSubscribeChannel.

Then, we declare a PubSubInboundChannelAdapter bean. It requires the channel we just created and a SubscriberFactory, which creates Subscriber objects from the Google Cloud Java Client for Pub/Sub. The Spring Boot starter for GCP Pub/Sub provides a configured SubscriberFactory.

It is also possible to set the message acknowledgement mode on the adapter, which is automatic by default. On automatic acking, a message is acked with GCP Pub/Sub if the adapter sent it to the channel and no exceptions were thrown. If a RuntimeException is thrown while the message is processed, then the message is nacked. On manual acking, the adapter attaches an AckReplyConsumer object to the Message headers, which users can extract using the GcpHeaders.ACKNOWLEDGEMENT key and use to (n)ack a message.

@Bean
@ServiceActivator(inputChannel = "pubsubInputChannel")
public MessageHandler messageReceiver() {
    return message -> {
        LOGGER.info("Message arrived! Payload: " + message.getPayload());
        AckReplyConsumer consumer =
              message.getHeaders().get(GcpHeaders.ACKNOWLEDGEMENT, AckReplyConsumer.class);
        consumer.ack();
    };
}

7.1.2 Outbound channel adapter

PubSubMessageHandler is the outbound channel adapter for GCP Pub/Sub that listens for new messages on a Spring MessageChannel. It uses PubSubTemplate to convert new Message instances to the GCP Pub/Sub format and post them to a GCP Pub/Sub topic.

To use the outbound channel adapter, a PubSubMessageHandler bean must be provided and configured on the user application side.

@Bean
@ServiceActivator(inputChannel = "pubsubOutputChannel")
public MessageHandler messageSender(PubSubTemplate pubsubTemplate) {
    return new PubSubMessageHandler(pubsubTemplate, "topicName");
}

The provided PubSubTemplate contains all the necessary configuration to publish messages to a GCP Pub/Sub topic.