Chapter 11. Testing OSGi based Applications

By following best practices and using the Spring Dynamic Modules support, your bean classes should be easy to unit test as they will have no hard dependencies on OSGi, and the few OSGi APIs that you may interact with (such as BundleContext) are interface-based and easy to mock. Whether you want to do unit testing or integration testing, Spring DM can ease your task.

11.1. OSGi Mocks

Even though most OSGi API are interfaces and creating mocks using a specialized library like EasyMock is fairly simple, in practice the amount of code of setting the code (especially on JDK 1.4) becomes cumbersome. To keep the tests short and concise, Spring DM provides OSGi mocks under org.springframework.osgi.mock package.

It's up to you to decide whether they are useful or not however, we make extensive use of them inside Spring DM test suite. Below you can find a code snippet that you are likely to encounter in our code base:

private ServiceReference reference;
private BundleContext bundleContext;
private Object service;
    
protected void setUp() throws Exception {
	reference = new MockServiceReference();
	bundleContext = new MockBundleContext() {

		public ServiceReference getServiceReference(String clazz) {
			return reference;
		}

		public ServiceReference[] getServiceReferences(String clazz, String filter) 
				throws InvalidSyntaxException {
			return new ServiceReference[] { reference };
		}
		
		public Object getService(ServiceReference ref) {
		    if (reference == ref)
		       return service;
		    super.getService(ref);
		}
	};

	...
}
	
public void testComponent() throws Exception {
    OsgiComponent comp = new OsgiComponent(bundleContext);
    
    assertSame(reference, comp.getReference());
    assertSame(object, comp.getTarget());
}

As ending words, experiment with them and choose whatever style or library you feel most confortable with. In our test suite we use the aforementioned mocks, EasyMock library and plenty of integration testing (see below).

11.2. Integration Testing

In a restricted environment such as OSGi, it's important to test the visibility and versioning of your classes, the manifests or how your bundles interact with each other (just to name a few).

To ease integration testing, the Spring Dynamic Modules project provides a test class hierarchy (based on org.springframework.osgi.test.AbstractOsgiTests) that provides support for writing regular JUnit test cases that are then automatically executed in an OSGi environment.

In general, the scenario supported by Spring DM testing framework is:

  • start the OSGi framework (Equinox, Knopflerfish, Felix)

  • install and start any specified bundles required for the test

  • package the test case itself into a on the fly bundle, generate the manifest (if none is provided) and install it in the OSGi framework

  • execute the test case inside the OSGi framework

  • shut down the framework

  • passes the test results back to the originating test case instance that is running outside of OSGi

[Warning]Warning
The testing framework is aimed at running OSGi integration tests from a non-OSGi environment (like Ant/Maven/IDE). The testing framework is NOT meant to be used as an OSGi bundle (nor will it work for that matter). In practice this means that the testing bundle should be separate from the bundle(s) it tests (similar to unit testing, where tests are separate from the classes they test).

By following this sequence it is trivial to write JUnit-based integration tests for OSGi and have them integration into any environment (IDE, build (ant, maven), etc.) that can work with JUnit.

The rest of this chapter details (with examples) the features offered by Spring DM testing suite.

11.2.1. Creating A Simple OSGi Integration Test

While the testing framework contains several classes that offer specific features, it is most likely that your test cases will extend org.springframework.osgi.test.AbstractConfigurableBundleCreatorTests (at least this is what we use in practice).

Let's extend this class and interact with the OSGi platform through the bundleContext field:

public class SimpleOsgiTest extends AbstractConfigurableBundleCreatorTests {

public void testOsgiPlatformStarts() throws Exception {
	System.out.println(bundleContext.getProperty(Constants.FRAMEWORK_VENDOR));
	System.out.println(bundleContext.getProperty(Constants.FRAMEWORK_VERSION));
	System.out.println(bundleContext.getProperty(Constants.FRAMEWORK_EXECUTIONENVIRONMENT));
	}
}

Simply execute the test as you normally do with any JUnit test. On Equinox 3.2.x, the output is similar to:

Eclipse
1.3.0
OSGi/Minimum-1.0,OSGi/Minimum-1.1,JRE-1.1,J2SE-1.2,J2SE-1.3,J2SE-1.4}

It is likely that you will see different log statements made by the testing framework during your own test execution, but these can be disabled as they only have an informative value and do not affect the actual execution.

Note that you did not have to create any bundle, write any MANIFEST or bother with imports or exports, let alone starting and shutting down the OSGi platform. The testing framework takes care of these automatically when the test is executed.

Let's do some quering and figure out what the environment in which the tests run is. A simple way to do that is to query the BundleContext for the installed bundles:

public void testOsgiEnvironment() throws Exception {
	Bundle[] bundles = bundleContext.getBundles();
	for (int i = 0; i < bundles.length; i++) {
		System.out.print(OsgiStringUtils.nullSafeName(bundles[i]));
		System.out.print(", ");
	}
	System.out.println();
}

The output should be similar to:

OSGi System Bundle, ObjectWeb ASM, log4j.osgi, spring-test, spring-osgi-test, spring-osgi-core, 
    spring-aop, spring-osgi-io, slf4j-api, 
spring-osgi-extender, etc... TestBundle-testOsgiPlatformStarts-com.your.package.SimpleOsgiTest, 

As you can see, the testing framework installs the mandatory requirements required for running the test such as the Spring, Spring DM, slf4j jars among others.

11.2.2. Installing Test Prerequisites

Besides the Spring DM jars and the test itself is highly likely that you depend on several libraries or your own code for the integration test.

Consider the following test that relies on Apache Commons Lang:

import org.apache.commons.lang.time.DateFormatUtils;
    ...
  	public void testCommonsLangDateFormat() throws Exception {
		System.out.println(DateFormatUtils.format(new Date(), "HH:mm:ssZZ"));
	}
}

Running the test however yields an exception:

java.lang.IllegalStateException: Unable to dynamically start generated unit test bundle
     ...
Caused by: org.osgi.framework.BundleException: The bundle could not be resolved. 
Reason: Missing Constraint: Import-Package: org.apache.commons.lang.time; version="0.0.0"
    ...
	... 15 more
	

The test requires org.apache.commons.lang.time package but there is no bundle that exports it. Let's fix this by installing a commons-lang bundle (make sure you use at least version 2.4 which adds the proper OSGi entries to the jar manifest).

One can specify the bundles that she wants to be installed using getTestBundlesNames or getTestBundles method. The first one returns an array of String that indicate the bundle name, package and versioning through as a String while the latter returns an array of Resources that can be used directly for installing the bundles. That is, use getTestBundlesNames when you rely on somebody else to locate (the most common case) the bundles and getTestBundles when you want to locate the bundles yourself.

By default, the test suite performs a lookup for artifacts, similar to the one used by Maven2, searching first the items as being relative to the current project and then falling back to the local repository. The locator expects the bundle String to be a comma separated values containing the artifact group, name, version and (optionally) type. It's likely that in the future, various other locators will be available. One can plug in their own locator through the org.springframework.osgi.test.provisioning.ArtifactLocator interface.

Let's fix our integration test by installing the required bundle (and some extra osgi libraries):

protected String[] getTestBundlesNames() {
	 return new String[] { "net.sourceforge.cglib, com.springsource.net.sf.cglib, 2.1.3",
	 	"javax.transaction, com.springsource.javax.transaction, 1.1.0",
	 	"commons-lang, commons-lang, 2.4" };
	 };
}

Rerunning the test should show that these bundles are now installed in the OSGi platform.

[Note]Note
The artifacts mentioned above have to exist in your local maven repository.

11.2.3. Advanced Testing Framework Topics

The testing framework allows a lot of customization to be made. This chapter details some of the existing hooks that you might want to know about. However, these are advanced topics as they increase the complexity of your test infrastructure.

11.2.3.1. Customizing The Test Manifest

There are cases where the auto-generated test manifest does not suite the needs of the test. For example the manifest requires some different headers or a certain package needs to be an optional import.

For simple cases, one can work directly with the generated manifest - in the example below, the bundle class path is being specified:

protected Manifest getManifest() {
      // let the testing framework create/load the manifest
      Manifest mf = super.getManifest();
      // add Bundle-Classpath:
      mf.getMainAttributes().putValue(Constants.BUNDLE_CLASSPATH, ".,bundleclasspath/simple.jar");
      return mf;
}

Another alternative is to provide your own manifest by overriding getManifestLocations():

protected String getManifestLocation() {
      return "classpath:com/xyz/abc/test/MyTestTest.MF";
}

However each manifest needs the following entry:

Bundle-Activator: org.springframework.osgi.test.JUnitTestActivator

since without it, the testing infrastructure cannot function properly. Also, one needs to import JUnit, Spring and Spring DM specific packages used by the base test suite:

Import-Package: junit.framework,
  org.osgi.framework,
  org.apache.commons.logging,
  org.springframework.util,
  org.springframework.osgi.service,
  org.springframework.osgi.util,
  org.springframework.osgi.test,
  org.springframework.context

Failing to import a package used by the test class will cause the test to fail with a NoDefClassFoundError error.

11.2.3.2. Customizing Test Bundle Content

By default, for the on-the-fly bundle, the testing infrastructure uses all the classes, xml and properties files found under ./target/test-classes folder. This matches the project layout for maven which is used (at the moment by Spring DM). These settings can be configured in two ways:

  1. programmatically by overriding AbstractConfigurableBundleCreatorTests getXXX methods.

  2. declaratively by creating a properties file having a similar name with the test case. For example, test com.xyz.MyTest will have the properties file named com/xyz/MyTest-bundle.properties. If found, the following properties will be read from the file:

    Table 11.1. Default test jar content settings

    Property NameDefault ValueDescription
    root.dirfile:./target/test-classesthe root folder considered as the jar root
    include.patterns/**/*.class,/**/*.xml,/**/*.propertiesComma-separated string of Ant-style patterns
    manifest(empty)manifest location given as a String. By default it's empty meaning the manifest will be created by the test framework rather then being supplied by the user.

    This option is handy when creating specific tests that need to include certain resources (such as localization files or images).

Please consult AbstractConfigurableBundleCreatorTests and AbstractOnTheFlyBundleCreatorTests tests for more customization hooks.

11.2.3.3. Understanding The MANIFEST.MF Generation

A useful feature of the testing framework represents the automatic creation of the test manifest based on the test bundle content. The manifest creator component uses byte-code analysis to determine the packages imported by the test classes so that it can generate the proper OSGi directives for them. Since the generated bundle is used for running a test, the creator will use the following assumptions:

  • No packages will be exported.

    The on-the-fly bundle is used for running a test which (usually) consumes OSGi packages for its execution. This behaviour can be changed by customizing the manifest.

  • Split packages (i.e. classes from the same package can come from different bundles) are not supported.

    This means that packages present in the test framework are considered complete and no Import-Package entry will be generated for them. To avoid this problem, consider using sub-packages or moving the classes inside one bundle. Note that split packages are discouraged due to the issues associated with them (see the OSGi Core spec, Chapter 3.13 - Required Bundles).

  • The test bundle contains only test classes.

    The byte-code parser will look only at the test classes hierarchy. Any other class included in the bundle, will not be considered so no imports will be generated for it. To change the default behaviour, override createManifestOnlyFromTestClass to return false:

    protected boolean createManifestOnlyFromTestClass() {
    	return false;
    }
    [Note]Note
    The time required to generate the manifest might increase depending on the number and size of classes in the bundle.

    Additionally consider customizing the manifest yourself or attaching the extra code as inner classes to the test class (so it gets parsed automatically).

The reason behind the lack of such features is the byte-code parser is aimed to be simple and fast at creating test manifests - it is not meant as a general-purpose tool for creating OSGi artifacts.

11.2.4. Creating An OSGi Application Context

Spring DM testing suite builds on top of Spring testing classes. To create an application context (OSGi specific), one should just override getConfigLocations[] method and indicate the location of the application context configuration. At runtime, an OSGi application context will be created and cached for the lifetime of the test case.

protected String[] getConfigLocations() {
   return new String[] { "/com/xyz/abc/test/MyTestContext.xml" };
}

11.2.5. Specifying The OSGi Platform To Use

The testing framework supports out of the box, three OSGi 4.0 implementations namely: Equinox, Knopflerfish and Felix. To be used, these should be in the test classpath. By default, the testing framework will try to use Equinox platform. This can be configured in several ways:

  1. programmatically through getPlatformName() method

    .

    Override the aforementioned method and indicate the fully qualified name of the Platform interface implementation. Users can use the Platforms class to specify one of the supported platforms:

    protected String getPlatformName() {
       return Platforms.FELIX;
    }
  2. declaratively through org.springframework.osgi.test.framework system property.

    If this property is set, the testing framework will use its value as a fully qualified name of a Platform implementation. It that fails, it will fall back to Equinox after logging a warning message. This option is useful for building tools (such as ant or maven) since it indicates a certain target environment without changing and test code.

11.2.6. Waiting For The Test Dependencies

A built-in feature of the testing framework is the ability to wait until all dependencies are deployed before starting the test execution. Since the OSGi platforms are concurrent by nature, installing a bundle doesn't mean that all its services are running. By running a test before its dependency services are fully initialized can cause sporadic errors that pollute the test results. By default, the testing framework inspects all bundles installed by the user and, if they are Spring-powered bundles, waits until they are fully started (that is their application context is published as an OSGi service). This behaviour can be disabled by overriding shouldWaitForSpringBundlesContextCreation method. Consult AbstractSynchronizedOsgiTests for more details.

11.2.7. Testing Framework Performance

Considering all the functionality offered by the testing framework, one might wonder if this doesn't become a performance bottleneck. First, it's worth noting that all the work done automatically by the testing infrastructure has to be done anyway (such as creating the manifest or creating a bundle for the test or installing the bundles). Doing it manually simply does not work as it's too error prone and time consuming. In fact, the current infrastructure started as way to do efficient, automatic testing without worrying about deployment problems and redundancy.

As for the numbers, the current infrastructure has been used internally for the last half a year - our integration tests (around 120) run in about 3:30 minutes on a laptop. Most of this time is spent on starting and stopping the OSGi platform: the "testing framework" takes around 10% (as shown in our profiling so far). For example, the manifest generation has proved to take less then 0.5 seconds in general, while the jar creation around 1 second.

However, we are working on making it even faster and smarter so that less configuration options are needed and the contextual information available in your tests is used as much as possible. If you have any ideas or suggestion, feel free to use our issue tracker or/and forum.

Hopefully this chapter showed how Spring DM testing infrastructure can simplify OSGi integration testing and how it can be customized. Consider consulting the javadocs for more information.