org.springframework.data.hadoop.mapreduce
Class JobRunner

java.lang.Object
  extended by org.springframework.data.hadoop.mapreduce.JobRunner
All Implemented Interfaces:
Callable<Void>, BeanFactoryAware, DisposableBean, InitializingBean

public class JobRunner
extends Object
implements Callable<Void>

Simple runner for submitting Hadoop jobs sequentially. By default, the runner waits for the jobs to finish and returns a boolean indicating whether all the jobs succeeded or not (when there's no waiting, the status cannot be determined and null is returned).

For more control over the job execution and outcome consider querying the Jobs or using Spring Batch (see the reference documentation for more info).

To make the runner execute at startup, use setRunAtStartup(boolean).

Author:
Costin Leau

Field Summary
protected  Log log
           
 
Constructor Summary
JobRunner()
           
 
Method Summary
 void afterPropertiesSet()
           
 Void call()
           
 void destroy()
           
protected  Collection<Job> findJobs()
           
 boolean isKillJobsAtShutdown()
          Indicates whether the configured jobs should be 'killed' when the application shuts down or not.
 boolean isVerbose()
          Indicates whether the job execution is verbose (the default) or not.
 boolean isWaitForCompletion()
          Indicates whether the 'runner' should wait for the job to complete (default).
 void setBeanFactory(BeanFactory beanFactory)
           
 void setExecutor(Executor executor)
          Sets the TaskExecutor used for executing the Hadoop job.
 void setJob(Job job)
          Sets the job to execute.
 void setJobNames(String... jobName)
          Sets the jobs to execute by (bean) name.
 void setJobs(Collection<Job> jobs)
          Sets the jobs to execute.
 void setKillJobAtShutdown(boolean killJobsAtShutdown)
          Indicates whether the configured jobs should be 'killed' when the application shuts down (default) or not.
 void setPostAction(Collection<Callable<?>> actions)
          Actions to be invoked after running the action.
 void setPreAction(Collection<Callable<?>> actions)
          Actions to be invoked before running the action.
 void setRunAtStartup(boolean runAtStartup)
          Indicates whether the jobs should be submitted at startup (default) or not.
 void setVerbose(boolean verbose)
          Indicates whether the job execution is verbose (the default) or not.
 void setWaitForCompletion(boolean waitForJob)
          Indicates whether the 'runner' should wait for the job to complete (default) after submission or not.
protected  Collection<Job> startJobs()
           
protected  Collection<Job> startJobs(org.springframework.data.hadoop.mapreduce.JobExecutor.JobListener listener)
           
protected  Collection<Job> stopJobs()
          Stops running job.
protected  Collection<Job> stopJobs(org.springframework.data.hadoop.mapreduce.JobExecutor.JobListener listener)
          Stops running job.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

log

protected Log log
Constructor Detail

JobRunner

public JobRunner()
Method Detail

afterPropertiesSet

public void afterPropertiesSet()
                        throws Exception
Specified by:
afterPropertiesSet in interface InitializingBean
Throws:
Exception

call

public Void call()
          throws Exception
Specified by:
call in interface Callable<Void>
Throws:
Exception

setRunAtStartup

public void setRunAtStartup(boolean runAtStartup)
Indicates whether the jobs should be submitted at startup (default) or not.

Parameters:
runAtStartup - The runAtStartup to set.

setPreAction

public void setPreAction(Collection<Callable<?>> actions)
Actions to be invoked before running the action.

Parameters:
actions -

setPostAction

public void setPostAction(Collection<Callable<?>> actions)
Actions to be invoked after running the action.

Parameters:
actions -

destroy

public void destroy()
             throws Exception
Specified by:
destroy in interface DisposableBean
Throws:
Exception

stopJobs

protected Collection<Job> stopJobs()
Stops running job.

Returns:
list of stopped jobs.
Throws:
Exception

stopJobs

protected Collection<Job> stopJobs(org.springframework.data.hadoop.mapreduce.JobExecutor.JobListener listener)
Stops running job.

Parameters:
listener - job listener
Returns:
list of stopped jobs.
Throws:
Exception

startJobs

protected Collection<Job> startJobs()

startJobs

protected Collection<Job> startJobs(org.springframework.data.hadoop.mapreduce.JobExecutor.JobListener listener)

findJobs

protected Collection<Job> findJobs()

setJob

public void setJob(Job job)
Sets the job to execute.

Parameters:
job - The job to execute.

setJobs

public void setJobs(Collection<Job> jobs)
Sets the jobs to execute.

Parameters:
jobs - The job to execute.

setJobNames

public void setJobNames(String... jobName)
Sets the jobs to execute by (bean) name. This is the default method used by the hdp name space to allow lazy initialization and potential scoping to kick in.

Parameters:
jobName - The job to execute.

isWaitForCompletion

public boolean isWaitForCompletion()
Indicates whether the 'runner' should wait for the job to complete (default).

Returns:
whether to wait for the job to complete or not.

setWaitForCompletion

public void setWaitForCompletion(boolean waitForJob)
Indicates whether the 'runner' should wait for the job to complete (default) after submission or not.

Parameters:
waitForJob - whether to wait for the job to complete or not.

isVerbose

public boolean isVerbose()
Indicates whether the job execution is verbose (the default) or not.

Returns:
whether the job execution is verbose or not.

setVerbose

public void setVerbose(boolean verbose)
Indicates whether the job execution is verbose (the default) or not.

Parameters:
verbose - whether the job execution is verbose or not.

setBeanFactory

public void setBeanFactory(BeanFactory beanFactory)
                    throws BeansException
Specified by:
setBeanFactory in interface BeanFactoryAware
Throws:
BeansException

setExecutor

public void setExecutor(Executor executor)
Sets the TaskExecutor used for executing the Hadoop job. By default, SimpleAsyncTaskExecutor is used, meaning a background thread is used. For a fine-tuned control, a dedicated Executor is recommended.

Parameters:
executor - the task executor to use execute the Hadoop job.

isKillJobsAtShutdown

public boolean isKillJobsAtShutdown()
Indicates whether the configured jobs should be 'killed' when the application shuts down or not.

Returns:
whether or not to kill the configured jobs at shutdown

setKillJobAtShutdown

public void setKillJobAtShutdown(boolean killJobsAtShutdown)
Indicates whether the configured jobs should be 'killed' when the application shuts down (default) or not. For long-running or fire-and-forget jobs that live beyond the starting application, set this to false. Note that if setWaitForCompletion(boolean) is true, this flag is considered to be true as otherwise the application cannot shut down (since it has to keep waiting for the job).

Parameters:
killJobsAtShutdown - whether or not to kill configured jobs when the application shuts down