40. Testing

Spring Boot provides a number of utilities and annotations to help when testing your application. Test support is provided by two modules; spring-boot-test contains core items, and spring-boot-test-autoconfigure supports auto-configuration for tests.

Most developers will just use the spring-boot-starter-test ‘Starter’ which imports both Spring Boot test modules as well has JUnit, AssertJ, Hamcrest and a number of other useful libraries.

40.1 Test scope dependencies

If you use the spring-boot-starter-test ‘Starter’ (in the test scope), you will find the following provided libraries:

  • JUnit — The de-facto standard for unit testing Java applications.
  • Spring Test & Spring Boot Test — Utilities and integration test support for Spring Boot applications.
  • AssertJ — A fluent assertion library.
  • Hamcrest — A library of matcher objects (also known as constraints or predicates).
  • Mockito — A Java mocking framework.
  • JSONassert — An assertion library for JSON.
  • JsonPath — XPath for JSON.

These are common libraries that we generally find useful when writing tests. You are free to add additional test dependencies of your own if these don’t suit your needs.

40.2 Testing Spring applications

One of the major advantages of dependency injection is that it should make your code easier to unit test. You can simply instantiate objects using the new operator without even involving Spring. You can also use mock objects instead of real dependencies.

Often you need to move beyond ‘unit testing’ and start ‘integration testing’ (with a Spring ApplicationContext actually involved in the process). It’s useful to be able to perform integration testing without requiring deployment of your application or needing to connect to other infrastructure.

The Spring Framework includes a dedicated test module for just such integration testing. You can declare a dependency directly to org.springframework:spring-test or use the spring-boot-starter-test ‘Starter’ to pull it in transitively.

If you have not used the spring-test module before you should start by reading the relevant section of the Spring Framework reference documentation.

40.3 Testing Spring Boot applications

A Spring Boot application is just a Spring ApplicationContext, so nothing very special has to be done to test it beyond what you would normally do with a vanilla Spring context. One thing to watch out for though is that the external properties, logging and other features of Spring Boot are only installed in the context by default if you use SpringApplication to create it.

Spring Boot provides a @SpringBootTest annotation which can be used as an alternative to the standard spring-test @ContextConfiguration annotation when you need Spring Boot features. The annotation works by creating the ApplicationContext used in your tests via SpringApplication.

You can use the webEnvironment attribute of @SpringBootTest to further refine how your tests will run:

  • MOCK — Loads a WebApplicationContext and provides a mock servlet environment. Embedded servlet containers are not started when using this annotation. If servlet APIs are not on your classpath this mode will transparently fallback to creating a regular non-web ApplicationContext. Can be used in conjunction with @AutoConfigureMockMvc for MockMvc-based testing of your application.
  • RANDOM_PORT — Loads an EmbeddedWebApplicationContext and provides a real servlet environment. Embedded servlet containers are started and listening on a random port.
  • DEFINED_PORT — Loads an EmbeddedWebApplicationContext and provides a real servlet environment. Embedded servlet containers are started and listening on a defined port (i.e from your application.properties or on the default port 8080).
  • NONE — Loads an ApplicationContext using SpringApplication but does not provide any servlet environment (mock or otherwise).
[Note]Note

If your test is @Transactional, it will rollback the transaction at the end of each test method by default. If you’re using this arrangement in combination with either RANDOM_PORT or DEFINED_PORT, any transaction initiated on the server won’t rollback as the test is running in a different thread than the server processing.

[Note]Note

In addition to @SpringBootTest a number of other annotations are also provided for testing more specific slices of an application. See below for details.

[Tip]Tip

Don’t forget to also add @RunWith(SpringRunner.class) to your test, otherwise the annotations will be ignored.

40.3.1 Detecting test configuration

If you’re familiar with the Spring Test Framework, you may be used to using @ContextConfiguration(classes=…​) in order to specify which Spring @Configuration to load. Alternatively, you might have often used nested @Configuration classes within your test.

When testing Spring Boot applications this is often not required. Spring Boot’s @*Test annotations will search for your primary configuration automatically whenever you don’t explicitly define one.

The search algorithm works up from the package that contains the test until it finds a @SpringBootApplication or @SpringBootConfiguration annotated class. As long as you’ve structured your code in a sensible way your main configuration is usually found.

If you want to customize the primary configuration, you can use a nested @TestConfiguration class. Unlike a nested @Configuration class which would be used instead of a your application’s primary configuration, a nested @TestConfiguration class will be used in addition to your application’s primary configuration.

[Note]Note

Spring’s test framework will cache application contexts between tests. Therefore, as long as your tests share the same configuration (no matter how it’s discovered), the potentially time consuming process of loading the context will only happen once.

40.3.2 Excluding test configuration

If your application uses component scanning, for example if you use @SpringBootApplication or @ComponentScan, you may find top-level configuration classes created only for specific tests accidentally get picked up everywhere.

As we have seen above, @TestConfiguration can be used on an inner class of a test to customize the primary configuration. When placed on a top-level class, @TestConfiguration indicates that classes in src/test/java should not be picked up by scanning. You can then import that class explicitly where it is required:

@RunWith(SpringRunner.class)
@SpringBootTest
@Import(MyTestsConfiguration.class)
public class MyTests {

    @Test
    public void exampleTest() {
        ...
    }

}
[Note]Note

If you directly use @ComponentScan (i.e. not via @SpringBootApplication) you will need to register the TypeExcludeFilter with it. See the Javadoc for details.

40.3.3 Working with random ports

If you need to start a full running server for tests, we recommend that you use random ports. If you use @SpringBootTest(webEnvironment=WebEnvironment.RANDOM_PORT) an available port will be picked at random each time your test runs.

The @LocalServerPort annotation can be used to inject the actual port used into your test. For convenience, tests that need to make REST calls to the started server can additionally @Autowire a TestRestTemplate which will resolve relative links to the running server.

import org.junit.Test;
import org.junit.runner.RunWith;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import static org.assertj.core.api.Assertions.assertThat;

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class RandomPortExampleTests {

	@Autowired
	private TestRestTemplate restTemplate;

	@Test
	public void exampleTest() {
		String body = this.restTemplate.getForObject("/", String.class);
		assertThat(body).isEqualTo("Hello World");
	}

}

40.3.4 Mocking and spying beans

It’s sometimes necessary to mock certain components within your application context when running tests. For example, you may have a facade over some remote service that’s unavailable during development. Mocking can also be useful when you want to simulate failures that might be hard to trigger in a real environment.

Spring Boot includes a @MockBean annotation that can be used to define a Mockito mock for a bean inside your ApplicationContext. You can use the annotation to add new beans, or replace a single existing bean definition. The annotation can be used directly on test classes, on fields within your test, or on @Configuration classes and fields. When used on a field, the instance of the created mock will also be injected. Mock beans are automatically reset after each test method.

Here’s a typical example where we replace an existing RemoteService bean with a mock implementation:

import org.junit.*;
import org.junit.runner.*;
import org.springframework.beans.factory.annotation.*;
import org.springframework.boot.test.context.*;
import org.springframework.boot.test.mock.mockito.*;
import org.springframework.test.context.junit4.*;

import static org.assertj.core.api.Assertions.*;
import static org.mockito.BDDMockito.*;

@RunWith(SpringRunner.class)
@SpringBootTest
public class MyTests {

    @MockBean
    private RemoteService remoteService;

    @Autowired
    private Reverser reverser;

    @Test
    public void exampleTest() {
        // RemoteService has been injected into the reverser bean
        given(this.remoteService.someCall()).willReturn("mock");
        String reverse = reverser.reverseSomeCall();
        assertThat(reverse).isEqualTo("kcom");
    }

}

Additionally you can also use @SpyBean to wrap any existing bean with a Mockito spy. See the Javadoc for full details.

40.3.5 Auto-configured tests

Spring Boot’s auto-configuration system works well for applications, but can sometimes be a little too much for tests. It’s often helpful to load only the parts of the configuration that are required to test a ‘slice’ of your application. For example, you might want to test that Spring MVC controllers are mapping URLs correctly, and you don’t want to involve database calls in those tests; or you might be wanting to test JPA entities, and you’re not interested in web layer when those tests run.

The spring-boot-test-autoconfigure module includes a number of annotations that can be used to automatically configure such ‘slices’. Each of them works in a similar way, providing a @…​Test annotation that loads the ApplicationContext and one or more @AutoConfigure…​ annotations that can be used to customize auto-configuration settings.

[Tip]Tip

It’s also possible to use the @AutoConfigure…​ annotations with the standard @SpringBootTest annotation. You can use this combination if you’re not interested in ‘slicing’ your application but you want some of the auto-configured test beans.

40.3.6 Auto-configured JSON tests

To test that Object JSON serialization and deserialization is working as expected you can use the @JsonTest annotation. @JsonTest will auto-configure Jackson ObjectMapper, any @JsonComponent beans and any Jackson Modules. It also configures Gson if you happen to be using that instead of, or as well as, Jackson. If you need to configure elements of the auto-configuration you can use the @AutoConfigureJsonTesters annotation.

Spring Boot includes AssertJ based helpers that work with the JSONassert and JsonPath libraries to check that JSON is as expected. The JacksonTester, GsonTester and BasicJsonTester classes can be used for Jackson, Gson and Strings respectively. Any helper fields on the test class can be @Autowired when using @JsonTest.

import org.junit.*;
import org.junit.runner.*;
import org.springframework.beans.factory.annotation.*;
import org.springframework.boot.test.autoconfigure.json.*;
import org.springframework.boot.test.context.*;
import org.springframework.boot.test.json.*;
import org.springframework.test.context.junit4.*;

import static org.assertj.core.api.Assertions.*;

@RunWith(SpringRunner.class)
@JsonTest
public class MyJsonTests {

    @Autowired
    private JacksonTester<VehicleDetails> json;

    @Test
    public void testSerialize() throws Exception {
        VehicleDetails details = new VehicleDetails("Honda", "Civic");
        // Assert against a `.json` file in the same package as the test
        assertThat(this.json.write(details)).isEqualToJson("expected.json");
        // Or use JSON path based assertions
        assertThat(this.json.write(details)).hasJsonPathStringValue("@.make");
        assertThat(this.json.write(details)).extractingJsonPathStringValue("@.make")
                .isEqualTo("Honda");
    }

    @Test
    public void testDeserialize() throws Exception {
        String content = "{\"make\":\"Ford\",\"model\":\"Focus\"}";
        assertThat(this.json.parse(content))
                .isEqualTo(new VehicleDetails("Ford", "Focus"));
        assertThat(this.json.parseObject(content).getMake()).isEqualTo("Ford");
    }

}
[Note]Note

JSON helper classes can also be used directly in standard unit tests. Simply call the initFields method of the helper in your @Before method if you aren’t using @JsonTest.

A list of the auto-configuration that is enabled by @JsonTest can be found in the appendix.

40.3.7 Auto-configured Spring MVC tests

To test Spring MVC controllers are working as expected you can use the @WebMvcTest annotation. @WebMvcTest will auto-configure the Spring MVC infrastructure and limit scanned beans to @Controller, @ControllerAdvice, @JsonComponent, Filter, WebMvcConfigurer and HandlerMethodArgumentResolver. Regular @Component beans will not be scanned when using this annotation.

Often @WebMvcTest will be limited to a single controller and used in combination with @MockBean to provide mock implementations for required collaborators.

@WebMvcTest also auto-configures MockMvc. Mock MVC offers a powerful way to quickly test MVC controllers without needing to start a full HTTP server.

[Tip]Tip

You can also auto-configure MockMvc in a non-@WebMvcTest (e.g. SpringBootTest) by annotating it with @AutoConfigureMockMvc.

import org.junit.*;
import org.junit.runner.*;
import org.springframework.beans.factory.annotation.*;
import org.springframework.boot.test.autoconfigure.web.servlet.*;
import org.springframework.boot.test.mock.mockito.*;

import static org.assertj.core.api.Assertions.*;
import static org.mockito.BDDMockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

@RunWith(SpringRunner.class)
@WebMvcTest(UserVehicleController.class)
public class MyControllerTests {

    @Autowired
    private MockMvc mvc;

    @MockBean
    private UserVehicleService userVehicleService;

    @Test
    public void testExample() throws Exception {
        given(this.userVehicleService.getVehicleDetails("sboot"))
                .willReturn(new VehicleDetails("Honda", "Civic"));
        this.mvc.perform(get("/sboot/vehicle").accept(MediaType.TEXT_PLAIN))
                .andExpect(status().isOk()).andExpect(content().string("Honda Civic"));
    }

}
[Tip]Tip

If you need to configure elements of the auto-configuration (for example when servlet filters should be applied) you can use attributes in the @AutoConfigureMockMvc annotation.

If you use HtmlUnit or Selenium, auto-configuration will also provide a WebClient bean and/or a WebDriver bean. Here is an example that uses HtmlUnit:

import com.gargoylesoftware.htmlunit.*;
import org.junit.*;
import org.junit.runner.*;
import org.springframework.beans.factory.annotation.*;
import org.springframework.boot.test.autoconfigure.web.servlet.*;
import org.springframework.boot.test.mock.mockito.*;

import static org.assertj.core.api.Assertions.*;
import static org.mockito.BDDMockito.*;

@RunWith(SpringRunner.class)
@WebMvcTest(UserVehicleController.class)
public class MyHtmlUnitTests {

    @Autowired
    private WebClient webClient;

    @MockBean
    private UserVehicleService userVehicleService;

    @Test
    public void testExample() throws Exception {
        given(this.userVehicleService.getVehicleDetails("sboot"))
                .willReturn(new VehicleDetails("Honda", "Civic"));
        HtmlPage page = this.webClient.getPage("/sboot/vehicle.html");
        assertThat(page.getBody().getTextContent()).isEqualTo("Honda Civic");
    }

}
[Note]Note

By default Spring Boot will put WebDriver beans in a special “scope” to ensure that the driver is quit after each test, and that a new instance is injected. If you don’t want this behavor you can add @Scope("singleton") to your WebDriver @Bean definition.

A list of the auto-configuration that is enabled by @WebMvcTest can be found in the appendix.

40.3.8 Auto-configured Data JPA tests

@DataJpaTest can be used if you want to test JPA applications. By default it will configure an in-memory embedded database, scan for @Entity classes and configure Spring Data JPA repositories. Regular @Component beans will not be loaded into the ApplicationContext.

Data JPA tests are transactional and rollback at the end of each test by default, see the docs.spring.io/spring/docs/4.3.9.RELEASE/spring-framework-reference/htmlsingle#testcontext-tx-enabling-transactions [relevant section] in the Spring Reference Documentation for more details. If that’s not what you want, you can disable transaction management for a test or for the whole class as follows:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

@RunWith(SpringRunner.class)
@DataJpaTest
@Transactional(propagation = Propagation.NOT_SUPPORTED)
public class ExampleNonTransactionalTests {

}

Data JPA tests may also inject a TestEntityManager bean which provides an alternative to the standard JPA EntityManager specifically designed for tests. If you want to use TestEntityManager outside of @DataJpaTests you can also use the @AutoConfigureTestEntityManager annotation. A JdbcTemplate is also available if you need that.

import org.junit.*;
import org.junit.runner.*;
import org.springframework.boot.test.autoconfigure.orm.jpa.*;

import static org.assertj.core.api.Assertions.*;

@RunWith(SpringRunner.class)
@DataJpaTest
public class ExampleRepositoryTests {

    @Autowired
    private TestEntityManager entityManager;

    @Autowired
    private UserRepository repository;

    @Test
    public void testExample() throws Exception {
        this.entityManager.persist(new User("sboot", "1234"));
        User user = this.repository.findByUsername("sboot");
        assertThat(user.getUsername()).isEqualTo("sboot");
        assertThat(user.getVin()).isEqualTo("1234");
    }

}

In-memory embedded databases generally work well for tests since they are fast and don’t require any developer installation. If, however, you prefer to run tests against a real database you can use the @AutoConfigureTestDatabase annotation:

@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureTestDatabase(replace=Replace.NONE)
public class ExampleRepositoryTests {

    // ...

}

A list of the auto-configuration that is enabled by @DataJpaTest can be found in the appendix.

40.3.9 Auto-configured REST clients

The @RestClientTest annotation can be used if you want to test REST clients. By default it will auto-configure Jackson and GSON support, configure a RestTemplateBuilder and add support for MockRestServiceServer. The specific beans that you want to test should be specified using value or components attribute of @RestClientTest:

@RunWith(SpringRunner.class)
@RestClientTest(RemoteVehicleDetailsService.class)
public class ExampleRestClientTest {

    @Autowired
    private RemoteVehicleDetailsService service;

    @Autowired
    private MockRestServiceServer server;

    @Test
    public void getVehicleDetailsWhenResultIsSuccessShouldReturnDetails()
            throws Exception {
        this.server.expect(requestTo("/greet/details"))
                .andRespond(withSuccess("hello", MediaType.TEXT_PLAIN));
        String greeting = this.service.callRestService();
        assertThat(greeting).isEqualTo("hello");
    }

}

A list of the auto-configuration that is enabled by @RestClientTest can be found in the appendix.

40.3.10 Auto-configured Spring REST Docs tests

The @AutoConfigureRestDocs annotation can be used if you want to use Spring REST Docs in your tests. It will automatically configure MockMvc to use Spring REST Docs and remove the need for Spring REST Docs' JUnit rule.

import org.junit.Test;
import org.junit.runner.RunWith;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.servlet.MockMvc;

import static org.springframework.restdocs.mockmvc.MockMvcRestDocumentation.document;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;

@RunWith(SpringRunner.class)
@WebMvcTest(UserController.class)
@AutoConfigureRestDocs("target/generated-snippets")
public class UserDocumentationTests {

    @Autowired
    private MockMvc mvc;

    @Test
    public void listUsers() throws Exception {
        this.mvc.perform(get("/users").accept(MediaType.TEXT_PLAIN))
                .andExpect(status().isOk())
                .andDo(document("list-users"));
    }

}

In addition to configuring the output directory, @AutoConfigureRestDocs can also configure the host, scheme, and port that will appear in any documented URIs. If you require more control over Spring REST Docs' configuration a RestDocsMockMvcConfigurationCustomizer bean can be used:

@TestConfiguration
static class CustomizationConfiguration
        implements RestDocsMockMvcConfigurationCustomizer {

    @Override
    public void customize(MockMvcRestDocumentationConfigurer configurer) {
        configurer.snippets().withTemplateFormat(TemplateFormats.markdown());
    }

}

If you want to make use of Spring REST Docs' support for a parameterized output directory, you can create a RestDocumentationResultHandler bean. The auto-configuration will call alwaysDo with this result handler, thereby causing each MockMvc call to automatically generate the default snippets:

@TestConfiguration
static class ResultHandlerConfiguration {

    @Bean
    public RestDocumentationResultHandler restDocumentation() {
        return MockMvcRestDocumentation.document("{method-name}");
    }

}

40.3.11 Using Spock to test Spring Boot applications

If you wish to use Spock to test a Spring Boot application you should add a dependency on Spock’s spock-spring module to your application’s build. spock-spring integrates Spring’s test framework into Spock. Exactly how you can use Spock to test a Spring Boot application depends on the version of Spock that you are using.

[Note]Note

Spring Boot provides dependency management for Spock 1.0. If you wish to use Spock 1.1 you should override the spock.version property in your build.gradle or pom.xml file.

When using Spock 1.1, the annotations described above can only be used and you can annotate your Specification with @SpringBootTest to suit the needs of your tests.

When using Spock 1.0, @SpringBootTest will not work for a web project. You need to use @SpringApplicationConfiguration and @WebIntegrationTest(randomPort = true). Being unable to use @SpringBootTest means that you also lose the auto-configured TestRestTemplate bean. You can create an equivalent bean yourself using the following configuration:

@Configuration
static class TestRestTemplateConfiguration {

    @Bean
    public TestRestTemplate testRestTemplate(
            ObjectProvider<RestTemplateBuilder> builderProvider,
            Environment environment) {
        RestTemplateBuilder builder = builderProvider.getIfAvailable();
        TestRestTemplate template = builder == null ? new TestRestTemplate()
                : new TestRestTemplate(builder.build());
        template.setUriTemplateHandler(new LocalHostUriTemplateHandler(environment));
        return template;
    }

}

40.4 Test utilities

A few test utility classes are packaged as part of spring-boot that are generally useful when testing your application.

40.4.1 ConfigFileApplicationContextInitializer

ConfigFileApplicationContextInitializer is an ApplicationContextInitializer that can apply to your tests to load Spring Boot application.properties files. You can use this when you don’t need the full features provided by @SpringBootTest.

@ContextConfiguration(classes = Config.class,
    initializers = ConfigFileApplicationContextInitializer.class)
[Note]Note

Using ConfigFileApplicationContextInitializer alone won’t provide support for @Value("${…​}") injection. Its only job is to ensure that application.properties files are loaded into Spring’s Environment. For @Value support you need to either additionally configure a PropertySourcesPlaceholderConfigurer or use @SpringBootTest where one will be auto-configured for you.

40.4.2 EnvironmentTestUtils

EnvironmentTestUtils allows you to quickly add properties to a ConfigurableEnvironment or ConfigurableApplicationContext. Simply call it with key=value strings:

EnvironmentTestUtils.addEnvironment(env, "org=Spring", "name=Boot");

40.4.3 OutputCapture

OutputCapture is a JUnit Rule that you can use to capture System.out and System.err output. Simply declare the capture as a @Rule then use toString() for assertions:

import org.junit.Rule;
import org.junit.Test;
import org.springframework.boot.test.rule.OutputCapture;

import static org.hamcrest.Matchers.*;
import static org.junit.Assert.*;

public class MyTest {

	@Rule
	public OutputCapture capture = new OutputCapture();

	@Test
	public void testName() throws Exception {
		System.out.println("Hello World!");
		assertThat(capture.toString(), containsString("World"));
	}

}

40.4.4 TestRestTemplate

TestRestTemplate is a convenience alternative to Spring’s RestTemplate that is useful in integration tests. You can get a vanilla template or one that sends Basic HTTP authentication (with a username and password). In either case the template will behave in a test-friendly way: not following redirects (so you can assert the response location), ignoring cookies (so the template is stateless), and not throwing exceptions on server-side errors. It is recommended, but not mandatory, to use Apache HTTP Client (version 4.3.2 or better), and if you have that on your classpath the TestRestTemplate will respond by configuring the client appropriately.

public class MyTest {

	private TestRestTemplate template = new TestRestTemplate();

	@Test
	public void testRequest() throws Exception {
		HttpHeaders headers = template.getForEntity("http://myhost.com", String.class).getHeaders();
		assertThat(headers.getLocation().toString(), containsString("myotherhost"));
	}

}

If you are using the @SpringBootTest annotation with WebEnvironment.RANDOM_PORT or WebEnvironment.DEFINED_PORT, you can just inject a fully configured TestRestTemplate and start using it. If necessary, additional customizations can be applied via the RestTemplateBuilder bean:

@RunWith(SpringRunner.class)
@SpringBootTest
public class MyTest {

	@Autowired
	private TestRestTemplate template;

	@Test
	public void testRequest() throws Exception {
		HttpHeaders headers = template.getForEntity("http://myhost.com", String.class).getHeaders();
		assertThat(headers.getLocation().toString(), containsString("myotherhost"));
	}

	@TestConfiguration
	static class Config {

		@Bean
		public RestTemplateBuilder restTemplateBuilder() {
			return new RestTemplateBuilder()
				.additionalMessageConverters(...)
				.customizers(...);
		}

	}

}