org.springframework.data.hadoop.cascading
Class HadoopFlowFactoryBean

java.lang.Object
  org.springframework.data.hadoop.cascading.HadoopFlowFactoryBean

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanNameAware, org.springframework.beans.factory.FactoryBean<cascading.flow.hadoop.HadoopFlow>, org.springframework.beans.factory.InitializingBean

public class HadoopFlowFactoryBean
extends java.lang.Object
implements org.springframework.beans.factory.BeanNameAware

Factory for declarative HadoopFlow creation. Usually used with a Cascade. Note the flow is not started.

Author:

Costin Leau

Constructor Summary

HadoopFlowFactoryBean()

Method Summary

void	afterPropertiesSet()
T	getObject()
java.lang.Class<?>	getObjectType()
boolean	isSingleton()
void	setAddCascadingJars(boolean addCascadingJars) Indicates whether the Cascading library jars should be added to the flow classpath (through the DistributedCache).
void	setBeanName(java.lang.String name)
void	setConfiguration(org.apache.hadoop.conf.Configuration configuration) Sets the configuration.
void	setDefinition(cascading.flow.FlowDef flowDef) Sets the flow definition.
void	setJar(org.springframework.core.io.Resource jar) Sets the job jar (which might not be on the classpath).
void	setJarByClass(java.lang.Class<?> jarClass) Determines the job jar (available on the classpath) based on the given class.
void	setJobPoolingInterval(java.lang.Long jobPoolingInterval) Sets the job pooling interval.
void	setListeners(java.util.Collection<cascading.flow.FlowListener> listeners) Sets the listeners.
void	setMaxConcurrentSteps(java.lang.Integer maxConcurrentSteps) Sets the max concurrent steps.
void	setPriority(java.lang.Integer priority) Sets the flow submit priority.
void	setProperties(java.util.Properties properties) Sets the properties.
void	setSink(cascading.tap.Tap sink)
void	setSinks(java.util.Map<java.lang.String,cascading.tap.Tap> sinks) Sets the sinks.
void	setSkipStrategy(cascading.flow.FlowSkipStrategy skipStrategy) Sets the skip strategy.
void	setSource(cascading.tap.Tap source) Sets the source.
void	setSources(java.util.Map<java.lang.String,cascading.tap.Tap> sources) Sets the sources.
void	setStepStrategy(cascading.flow.FlowStepStrategy stepStrategy) Sets the step strategy.
void	setTail(cascading.pipe.Pipe tail) Sets the tail.
void	setTails(java.util.Collection<cascading.pipe.Pipe> tails) Sets the tails.
void	setTraps(java.util.Map<java.lang.String,cascading.tap.Tap> traps) Sets the traps.
void	setWriteDOT(java.lang.String writeDOT) Sets the write dot.
void	setWriteStepsDOT(java.lang.String writeStepsDOT) Sets the write steps dot.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopFlowFactoryBean

public HadoopFlowFactoryBean()

Method Detail

setBeanName

public void setBeanName(java.lang.String name)

Specified by:

setBeanName in interface org.springframework.beans.factory.BeanNameAware

setConfiguration

public void setConfiguration(org.apache.hadoop.conf.Configuration configuration)

Sets the configuration.

Parameters:

configuration - The configuration to set.

setProperties

public void setProperties(java.util.Properties properties)

Sets the properties.

Parameters:

properties - The properties to set.

setMaxConcurrentSteps

public void setMaxConcurrentSteps(java.lang.Integer maxConcurrentSteps)

Sets the max concurrent steps.

Parameters:

maxConcurrentSteps - The maxConcurrentSteps to set.

setJobPoolingInterval

public void setJobPoolingInterval(java.lang.Long jobPoolingInterval)

Sets the job pooling interval.

Parameters:

jobPoolingInterval - The jobPoolingInterval to set.

setSources

public void setSources(java.util.Map<java.lang.String,cascading.tap.Tap> sources)

Sets the sources.

Parameters:

sources - The sources to set.

setSource

public void setSource(cascading.tap.Tap source)

Sets the source.

Parameters:

source - The source to set.

setSinks

public void setSinks(java.util.Map<java.lang.String,cascading.tap.Tap> sinks)

Sets the sinks.

Parameters:

sinks - The sinks to set.

setSink

public void setSink(cascading.tap.Tap sink)

setTraps

public void setTraps(java.util.Map<java.lang.String,cascading.tap.Tap> traps)

Sets the traps.

Parameters:

traps - The traps to set.

setTails

public void setTails(java.util.Collection<cascading.pipe.Pipe> tails)

Sets the tails.

Parameters:

tails - The tails to set.

setTail

public void setTail(cascading.pipe.Pipe tail)

Sets the tail.

Parameters:

tail - the new tail

setDefinition

public void setDefinition(cascading.flow.FlowDef flowDef)

Sets the flow definition. Useful for programatically setting up the flow and then letting Spring do the configuration wiring.

Parameters:

flowDef -

setJarByClass

public void setJarByClass(java.lang.Class<?> jarClass)

Determines the job jar (available on the classpath) based on the given class.

Parameters:

jarClass - The jarClass to set.

setJar

public void setJar(org.springframework.core.io.Resource jar)

Sets the job jar (which might not be on the classpath).

Parameters:

jar - The jar to set.

setAddCascadingJars

public void setAddCascadingJars(boolean addCascadingJars)

Indicates whether the Cascading library jars should be added to the flow classpath (through the DistributedCache). By default it is true. When running against a cluster where Cascading jars are already installed, turn this to false to avoid shipping the library jars with the job.

Parameters:

addCascadingJars - whether the Cascading library jars should be added to the flow classpath or not

getObject

public T getObject()
                                           throws java.lang.Exception

Specified by:

getObject in interface org.springframework.beans.factory.FactoryBean<T extends cascading.flow.Flow<?>>

Throws:

java.lang.Exception

getObjectType

public java.lang.Class<?> getObjectType()

Specified by:

getObjectType in interface org.springframework.beans.factory.FactoryBean<T extends cascading.flow.Flow<?>>

isSingleton

public boolean isSingleton()

Specified by:

isSingleton in interface org.springframework.beans.factory.FactoryBean<T extends cascading.flow.Flow<?>>

afterPropertiesSet

public void afterPropertiesSet()
                        throws java.lang.Exception

Specified by:

afterPropertiesSet in interface org.springframework.beans.factory.InitializingBean

Throws:

java.lang.Exception

setSkipStrategy

public void setSkipStrategy(cascading.flow.FlowSkipStrategy skipStrategy)

Sets the skip strategy.

Parameters:

skipStrategy - The skipStrategy to set.

setStepStrategy

public void setStepStrategy(cascading.flow.FlowStepStrategy stepStrategy)

Sets the step strategy.

Parameters:

stepStrategy - The stepStrategy to set.

setListeners

public void setListeners(java.util.Collection<cascading.flow.FlowListener> listeners)

Sets the listeners.

Parameters:

listeners - The listeners to set.

setWriteDOT

public void setWriteDOT(java.lang.String writeDOT)

Sets the write dot.

Parameters:

writeDOT - the new write dot

setWriteStepsDOT

public void setWriteStepsDOT(java.lang.String writeStepsDOT)

Sets the write steps dot.

Parameters:

writeStepsDOT - the new write steps dot

setPriority

public void setPriority(java.lang.Integer priority)

Sets the flow submit priority.

Parameters:

priority - flow priority