org.springframework.data.hadoop.mapreduce
Class ToolRunner

java.lang.Object
  org.springframework.data.hadoop.mapreduce.ToolRunner

All Implemented Interfaces:

Callable<Integer>, BeanClassLoaderAware, InitializingBean

public class ToolRunner
extends Object
implements Callable<Integer>, InitializingBean

Wrapper around ToolRunner allowing for an easier configuration and execution of Tool instances inside Spring. Optionally returns the execution result (as an int per Tool.run(String[])).

To make the runner execute at startup, use setRunAtStartup(boolean).

Author:

Costin Leau

Constructor Summary

ToolRunner()

Method Summary

void	afterPropertiesSet()
Integer	call()
protected ClassLoader	createClassLoaderForJar(Resource jar, ClassLoader parentCL, Configuration cfg)
protected Integer	invokeTargetObject(Configuration cfg, Tool target, Class<Tool> targetClass, String[] args)
protected Class<Tool>	loadClass(String className, ClassLoader cl)
protected void	postExecution(Configuration cfg)
protected void	preExecution(Configuration cfg)
protected Configuration	resolveConfiguration()
protected Class<T>	resolveTargetClass(Configuration cfg)
protected T	resolveTargetObject(Class<T> type)
void	setArchives(Resource... archives) Sets the archives to be unarchive to the map reduce cluster.
void	setArguments(String... arguments) Sets the arguments.
void	setBeanClassLoader(ClassLoader classLoader)
void	setCloseFs(boolean closeFs) Indicates whether or not to close the Hadoop file-systems resulting from the custom code execution.
void	setConfiguration(Configuration configuration) Sets the configuration.
void	setFiles(Resource... files) Sets the files to be copied to the map reduce cluster.
void	setJar(Resource jar) Sets the target code jar.
void	setLibs(Resource... libJars) Sets the jar files to include in the classpath.
void	setPostAction(Collection<Callable<?>> actions) Actions to be invoked after running the action.
void	setPreAction(Collection<Callable<?>> actions) Actions to be invoked before running the action.
void	setProperties(Properties properties) Sets the properties.
void	setRunAtStartup(boolean runAtStartup) Indicates whether the tool should run at container startup (the default) or not.
void	setTool(Tool tool) Sets the tool.
void	setToolClass(String toolClassName) Sets the tool class by name.
void	setUser(String user) Sets the user impersonation (optional) for running this job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ToolRunner

public ToolRunner()

Method Detail

call

public Integer call()
             throws Exception

Specified by:

call in interface Callable<Integer>

Throws:

Exception

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception

Specified by:

afterPropertiesSet in interface InitializingBean

Throws:

Exception

setRunAtStartup

public void setRunAtStartup(boolean runAtStartup)

Indicates whether the tool should run at container startup (the default) or not.

Parameters:

runAtStartup - The runAtStartup to set.

setPreAction

public void setPreAction(Collection<Callable<?>> actions)

Actions to be invoked before running the action.

Parameters:

actions -

setPostAction

public void setPostAction(Collection<Callable<?>> actions)

Actions to be invoked after running the action.

Parameters:

actions -

invokeTargetObject

protected Integer invokeTargetObject(Configuration cfg,
                                     Tool target,
                                     Class<Tool> targetClass,
                                     String[] args)
                              throws Exception

Throws:

Exception

loadClass

protected Class<Tool> loadClass(String className,
                                ClassLoader cl)

setTool

public void setTool(Tool tool)

Sets the tool.

Parameters:

tool - The tool to set.

setToolClass

public void setToolClass(String toolClassName)

Sets the tool class by name.

Parameters:

toolClassName - the new tool class

resolveConfiguration

protected Configuration resolveConfiguration()
                                      throws Exception

Throws:

Exception

resolveTargetClass

protected Class<T> resolveTargetClass(Configuration cfg)
                               throws Exception

Throws:

Exception

resolveTargetObject

protected T resolveTargetObject(Class<T> type)

createClassLoaderForJar

protected ClassLoader createClassLoaderForJar(Resource jar,
                                              ClassLoader parentCL,
                                              Configuration cfg)

preExecution

protected void preExecution(Configuration cfg)

postExecution

protected void postExecution(Configuration cfg)

setJar

public void setJar(Resource jar)

Sets the target code jar.

Parameters:

jar -

setArguments

public void setArguments(String... arguments)

Sets the arguments.

Parameters:

arguments - The arguments to set.

setConfiguration

public void setConfiguration(Configuration configuration)

Sets the configuration.

Parameters:

configuration - The configuration to set.

setProperties

public void setProperties(Properties properties)

Sets the properties.

Parameters:

properties - The properties to set.

setBeanClassLoader

public void setBeanClassLoader(ClassLoader classLoader)

Specified by:

setBeanClassLoader in interface BeanClassLoaderAware

setCloseFs

public void setCloseFs(boolean closeFs)

Indicates whether or not to close the Hadoop file-systems resulting from the custom code execution. Default is true. Turn this to false if the code reuses the same file-system used by the rest of the application.

Parameters:

closeFs - the new close fs

setLibs

public void setLibs(Resource... libJars)

Sets the jar files to include in the classpath. Note that a pattern can be used (e.g. mydir/*.jar), which the Spring container will automatically resolve.

Parameters:

libJars - The jar files to include in the classpath.

setFiles

public void setFiles(Resource... files)

Sets the files to be copied to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.txt), which the Spring container will automatically resolve.

Parameters:

files - The files to copy.

setArchives

public void setArchives(Resource... archives)

Sets the archives to be unarchive to the map reduce cluster. Note that a pattern can be used (e.g. mydir/*.zip), which the Spring container will automatically resolve.

Parameters:

archives - The archives to unarchive on the compute machines.

setUser

public void setUser(String user)

Sets the user impersonation (optional) for running this job. Should be used when running against a Hadoop Kerberos cluster.

Parameters:

user - user/group information