org.springframework.data.hadoop.mapreduce
Class JobFactoryBean

java.lang.Object
  org.springframework.data.hadoop.mapreduce.JobFactoryBean

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanClassLoaderAware, org.springframework.beans.factory.BeanNameAware, org.springframework.beans.factory.FactoryBean<org.apache.hadoop.mapreduce.Job>, org.springframework.beans.factory.InitializingBean

public class JobFactoryBean
extends java.lang.Object
implements org.springframework.beans.factory.InitializingBean, org.springframework.beans.factory.FactoryBean<org.apache.hadoop.mapreduce.Job>, org.springframework.beans.factory.BeanNameAware, org.springframework.beans.factory.BeanClassLoaderAware

Factory bean for creating a Hadoop Map-Reduce job. Note that the setters for the class access class names (String) instead of direct classes. This is done on purpose in case the job itself and its dependencies are on a jar not available on the classpath (setJar(Resource)) in which case a special, on-the-fly classloader is used.

Author:

Costin Leau

Constructor Summary

JobFactoryBean()

Method Summary

void	afterPropertiesSet()
org.apache.hadoop.mapreduce.Job	getObject()
java.lang.Class<?>	getObjectType()
boolean	isSingleton()
protected void	processJob(org.apache.hadoop.mapreduce.Job job)
void	setArchives(org.springframework.core.io.Resource... archives) Sets the archives to be unarchive to the map reduce cluster.
void	setBeanClassLoader(java.lang.ClassLoader classLoader)
void	setBeanName(java.lang.String name)
void	setCodec(java.lang.String codecClass) Sets the job codec class.
void	setCombiner(java.lang.String combiner) Sets the job combiner class.
void	setCompressOutput(java.lang.Boolean compressOutput) Indicates whether the job output should be compressed or not.
void	setConfiguration(org.apache.hadoop.conf.Configuration configuration) Sets the Hadoop configuration to use.
void	setFiles(org.springframework.core.io.Resource... files) Sets the files to be copied to the map reduce cluster.
void	setGroupingComparator(java.lang.String groupingComparator) Sets the job grouping comparator class.
void	setInputFormat(java.lang.String inputFormat) Sets the job input format class.
void	setInputPath(java.lang.String... inputPath) Sets the job input path.
void	setJar(org.springframework.core.io.Resource jar) Sets the job jar (which might not be on the classpath).
void	setJarByClass(java.lang.Class<?> jarClass) Determines the job jar (available on the classpath) based on the given class.
void	setKey(java.lang.String key) Sets the job key class.
void	setLibs(org.springframework.core.io.Resource... libJars) Sets the jar files to include in the classpath.
void	setMapKey(java.lang.String mapKey) Sets the job map key class.
void	setMapper(java.lang.String mapper) Sets the job mapper class.
void	setMapValue(java.lang.String mapValue) Sets the job map value class.
void	setName(java.lang.String name) Sets the job name.
void	setNumberReducers(java.lang.Integer numReduceTasks) Sets the number of reduce task for this job.
void	setOutputFormat(java.lang.String outputFormat) Sets the job output format class.
void	setOutputPath(java.lang.String outputPath) Sets the job output path.
void	setPartitioner(java.lang.String partitioner) Sets the job partitioner class.
void	setProperties(java.util.Properties properties) The configuration properties to set for this job.
void	setReducer(java.lang.String reducer) Sets the job reducer class.
void	setSortComparator(java.lang.String sortComparator) Sets the job sort comparator class.
void	setUser(java.lang.String user) Sets the user impersonation (optional) for running this job.
void	setValue(java.lang.String value) Sets the job value class.
void	setWorkingDir(java.lang.String workingDir) Sets the job working directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobFactoryBean

public JobFactoryBean()

Method Detail

setBeanClassLoader

public void setBeanClassLoader(java.lang.ClassLoader classLoader)

Specified by:

setBeanClassLoader in interface org.springframework.beans.factory.BeanClassLoaderAware

setBeanName

public void setBeanName(java.lang.String name)

Specified by:

setBeanName in interface org.springframework.beans.factory.BeanNameAware

getObject

public org.apache.hadoop.mapreduce.Job getObject()
                                          throws java.lang.Exception

Specified by:

getObject in interface org.springframework.beans.factory.FactoryBean<org.apache.hadoop.mapreduce.Job>

Throws:

java.lang.Exception

getObjectType

public java.lang.Class<?> getObjectType()

Specified by:

getObjectType in interface org.springframework.beans.factory.FactoryBean<org.apache.hadoop.mapreduce.Job>

isSingleton

public boolean isSingleton()

Specified by:

isSingleton in interface org.springframework.beans.factory.FactoryBean<org.apache.hadoop.mapreduce.Job>

afterPropertiesSet

public void afterPropertiesSet()
                        throws java.lang.Exception

Specified by:

afterPropertiesSet in interface org.springframework.beans.factory.InitializingBean

Throws:

java.lang.Exception

processJob

protected void processJob(org.apache.hadoop.mapreduce.Job job)
                   throws java.lang.Exception

Throws:

java.lang.Exception

setConfiguration

public void setConfiguration(org.apache.hadoop.conf.Configuration configuration)

Sets the Hadoop configuration to use.

Parameters:

configuration - The configuration to set.

setName

public void setName(java.lang.String name)

Sets the job name.

Parameters:

name - The name to set.

setKey

public void setKey(java.lang.String key)

Sets the job key class.

Parameters:

key - The keyClass to set.

setValue

public void setValue(java.lang.String value)

Sets the job value class.

Parameters:

value - The valueClass to set.

setMapKey

public void setMapKey(java.lang.String mapKey)

Sets the job map key class.

Parameters:

mapKey - The mapKeyClass to set.

setMapValue

public void setMapValue(java.lang.String mapValue)

Sets the job map value class.

Parameters:

mapValue - The mapValueClass to set.

setMapper

public void setMapper(java.lang.String mapper)

Sets the job mapper class.

Parameters:

mapper - The mapper to set.

setReducer

public void setReducer(java.lang.String reducer)

Sets the job reducer class.

Parameters:

reducer - The reducer to set.

setCombiner

public void setCombiner(java.lang.String combiner)

Sets the job combiner class.

Parameters:

combiner - The combiner to set.

setInputFormat

public void setInputFormat(java.lang.String inputFormat)

Sets the job input format class.

Parameters:

inputFormat - The inputFormat to set.

setOutputFormat

public void setOutputFormat(java.lang.String outputFormat)

Sets the job output format class.

Parameters:

outputFormat - The outputFormat to set.

setPartitioner

public void setPartitioner(java.lang.String partitioner)

Sets the job partitioner class.

Parameters:

partitioner - The partitioner to set.

setSortComparator

public void setSortComparator(java.lang.String sortComparator)

Sets the job sort comparator class.

Parameters:

sortComparator - The sortComparator to set.

setGroupingComparator

public void setGroupingComparator(java.lang.String groupingComparator)

Sets the job grouping comparator class.

Parameters:

groupingComparator - The groupingComparator to set.

setWorkingDir

public void setWorkingDir(java.lang.String workingDir)

Sets the job working directory.

Parameters:

workingDir - The workingDir to set.

setNumberReducers

public void setNumberReducers(java.lang.Integer numReduceTasks)

Sets the number of reduce task for this job.

Parameters:

numReduceTasks - The numReduceTasks to set.

setJarByClass

public void setJarByClass(java.lang.Class<?> jarClass)

Determines the job jar (available on the classpath) based on the given class.

Parameters:

jarClass - The jarClass to set.

setJar

public void setJar(org.springframework.core.io.Resource jar)

Sets the job jar (which might not be on the classpath).

Parameters:

jar - The jar to set.

setInputPath

public void setInputPath(java.lang.String... inputPath)

Sets the job input path.

Parameters:

inputPath - job input path.

setOutputPath

public void setOutputPath(java.lang.String outputPath)

Sets the job output path.

Parameters:

outputPath - The outputPath to set.

setCompressOutput

public void setCompressOutput(java.lang.Boolean compressOutput)

Indicates whether the job output should be compressed or not.

Parameters:

compressOutput - The compressOutput to set.

setCodec

public void setCodec(java.lang.String codecClass)

Sets the job codec class.

Parameters:

codecClass - The codecClass to set.

setProperties

public void setProperties(java.util.Properties properties)

The configuration properties to set for this job.

Parameters:

properties - The properties to set.

setLibs

public void setLibs(org.springframework.core.io.Resource... libJars)

Sets the jar files to include in the classpath. Note that a pattern can be used (e.g. mydir/*.jar), which the Spring container will automatically resolve.

Parameters:

libJars - The jar files to include in the classpath.

setFiles

public void setFiles(org.springframework.core.io.Resource... files)

Sets the files to be copied to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.txt), which the Spring container will automatically resolve.

Parameters:

files - The files to copy.

setArchives

public void setArchives(org.springframework.core.io.Resource... archives)

Sets the archives to be unarchive to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.zip), which the Spring container will automatically resolve.

Parameters:

archives - The archives to unarchive on the compute machines.

setUser

public void setUser(java.lang.String user)

Sets the user impersonation (optional) for running this job. Should be used when running against a Hadoop Kerberos cluster.

Parameters:

user - user/group information