org.springframework.data.hadoop.test.junit

Class AbstractMapReduceTests

java.lang.Object

org.springframework.data.hadoop.test.junit.AbstractHadoopClusterTests

org.springframework.data.hadoop.test.junit.AbstractMapReduceTests

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.context.ApplicationContextAware

public class AbstractMapReduceTests
extends AbstractHadoopClusterTests

Abstract base class providing default functionality for running map reduce tests.

Author:

Janne Valkealahti

Field Summary

Fields inherited from class org.springframework.data.hadoop.test.junit.AbstractHadoopClusterTests

applicationContext, configuration, hadoopCluster

Constructor Summary

Constructors

Constructor and Description
AbstractMapReduceTests()

Method Summary

Methods

Modifier and Type	Method and Description
protected JobUtils.JobStatus	findStatus(org.apache.hadoop.mapreduce.Job job) Find job status.
protected org.apache.hadoop.fs.Path[]	getOutputFilePaths(org.apache.hadoop.fs.Path outputDirectory) Finds an array of output file Paths resulted as mapreduce job run.
protected org.apache.hadoop.fs.Path[]	getOutputFilePaths(java.lang.String outputDirectory) Finds an array of output file Paths resulted as mapreduce job run.
protected JobUtils.JobStatus	waitFinishedStatus(org.apache.hadoop.mapreduce.Job job, long timeout, java.util.concurrent.TimeUnit unit) Wait finished status.
protected JobUtils.JobStatus	waitStatus(org.apache.hadoop.mapreduce.Job job, long timeout, java.util.concurrent.TimeUnit unit, JobUtils.JobStatus... jobStatuses) Waits state.

Methods inherited from class org.springframework.data.hadoop.test.junit.AbstractHadoopClusterTests

getApplicationContext, getConfiguration, getFileSystem, getHadoopCluster, setApplicationContext, setConfiguration, setHadoopCluster

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractMapReduceTests

public AbstractMapReduceTests()

Method Detail

getOutputFilePaths

protected org.apache.hadoop.fs.Path[] getOutputFilePaths(java.lang.String outputDirectory)
                                                  throws java.io.FileNotFoundException,
                                                         java.io.IOException

Finds an array of output file Paths resulted as mapreduce job run.

Parameters:

outputDirectory - the path to jobs output directory

Returns:

list of output files

Throws:

java.io.FileNotFoundException - if given path was not found

java.io.IOException - if general access error occured

See Also:

getOutputFilePaths(Path)

getOutputFilePaths

protected org.apache.hadoop.fs.Path[] getOutputFilePaths(org.apache.hadoop.fs.Path outputDirectory)
                                                  throws java.io.FileNotFoundException,
                                                         java.io.IOException

Finds an array of output file Paths resulted as mapreduce job run.

Parameters:

outputDirectory - the path to jobs output directory

Returns:

list of output files

Throws:

java.io.FileNotFoundException - if given path was not found

java.io.IOException - if general access error occured

waitFinishedStatus

protected JobUtils.JobStatus waitFinishedStatus(org.apache.hadoop.mapreduce.Job job,
                                    long timeout,
                                    java.util.concurrent.TimeUnit unit)
                                         throws java.lang.Exception

Wait finished status. Statuses to wait is one of following, JobStatus.SUCCEEDED, JobStatus.FAILED or JobStatus.KILLED.

Parameters:

job - the job

timeout - the timeout for wait

unit - the unit for timeout

Returns:

the job status

Throws:

java.lang.Exception - if exception occurred

waitStatus

protected JobUtils.JobStatus waitStatus(org.apache.hadoop.mapreduce.Job job,
                            long timeout,
                            java.util.concurrent.TimeUnit unit,
                            JobUtils.JobStatus... jobStatuses)
                                 throws java.lang.Exception

Waits state. Returned state is NULL if something failed or final known state after the wait/poll operations. Array of job statuses can be used to return immediately from wait loop if state is matched.

Parameters:

job - the job

timeout - the timeout for wait

unit - the unit for timeout

jobStatuses - the job statuses to wait

Returns:

the job status

Throws:

java.lang.Exception - if exception occurred

findStatus

protected JobUtils.JobStatus findStatus(org.apache.hadoop.mapreduce.Job job)

Find job status.

Parameters:

job - the job

Returns:

the job status