Spring for Apache Hadoop 2.0 is built and tested with JDK 7.0 (generated jars are usable in JDK 6.0 and above), Spring Framework 4.0 and is by default built against Apache Hadoop 2.2.0.
Spring for Apache Hadoop 2.0 supports the following versions and distributions:
Apache Hadoop 1.2.1
Apache Hadoop 2.2.0 *
Apache Hadoop 2.4.0 *
Pivotal HD 1.1
Pivotal HD 2.0 *
Cloudera CDH4 (CDH4.6.0)
Cloudera CDH5 (2.3.0-cdh5.0.0) *
Hortonworks Data Platform 1.3
Hortonworks Data Platform 2.0 *
Hortonworks Data Platform 2.1 *
* The distributions noted with and asterisk will include spring-yarn support in the build.
Any distribution compatible with Apache Hadoop 1.x or 2.2.x should be supported.
Note | |
---|---|
Spring for Apache Hadoop has been certified to work on Pivotal HD 1.0 and 1.1, Hortonworks HDP 1.3 and Cloudera CDH 4.4 distributions. Further certifications will be done soon now that Spring for Apache 2.0 has reached GA. |
Spring for Apache Hadoop 2.0 is tested daily against a number of Hadoop distributions. See the test plan page for current status.
Instructions for setting up project builds using various supported distributions are provided on the Spring for Apache Hadoop wiki - https://github.com/spring-projects/spring-hadoop/wiki
Regarding Hadoop-related projects, SDHP supports HBase 0.94.11, Hive 0.10.0 and Pig 0.10.1 and above. As a rule of thumb, when using Hadoop-related projects, such as Hive or Pig, use the required Hadoop version as a basis for discovering the supported versions.
To take full advantage of Spring for Apache Hadoop you need a running Hadoop cluster. If you don't already have one in your environment, a good first step is to create a single-node cluster. To install the most recent stable verision of Hadoop, the "Getting Started" page from the official Apache documentation is a good general guide. There should be a link for "Single Node Setup".
It is also convenient to download a Virtual Machine where Hadoop is setup and ready to go. Cloudera, Hortonworks and Pivotal all provide virtual machines and provide VM downloads on their product pages. Additionally, the appendix provides information on how to use Spring for Apache Hadoop and setup Hadoop with cloud providers, such as Amazon Web Services.