org.springframework.yarn.batch.partition

Class AbstractPartitioner

java.lang.Object

org.springframework.yarn.batch.partition.AbstractPartitioner

All Implemented Interfaces:

org.springframework.batch.core.partition.support.Partitioner

Direct Known Subclasses:

SplitterPartitioner

public abstract class AbstractPartitioner
extends java.lang.Object
implements org.springframework.batch.core.partition.support.Partitioner

Base class providing share functionality for batch Partitioner.

Author:

Janne Valkealahti

Constructor Summary

Constructors

Constructor and Description
AbstractPartitioner()

Method Summary

Methods

Modifier and Type	Method and Description
protected org.springframework.batch.item.ExecutionContext	createExecutionContext(org.apache.hadoop.fs.Path path, Split split) Creates the execution context.
protected org.springframework.batch.item.ExecutionContext	createExecutionContext(org.springframework.core.io.Resource resource, Split split) Creates the execution context.
protected abstract java.util.Map<java.lang.String,org.springframework.batch.item.ExecutionContext>	createPartitions() Creates the ExecutionContexts for partitions.
org.apache.hadoop.conf.Configuration	getConfiguration() Gets the hadoop configuration.
java.lang.String	getKeyFileName() Gets the key for a file name.
java.lang.String	getKeySplitLength() Gets the key for split length.
java.lang.String	getKeySplitLocations() Gets the key for split locations.
java.lang.String	getKeySplitStart() Gets the key for split start.
java.lang.String	getPartitionBaseIdentifier() Gets the partition base identifier.
java.util.Map<java.lang.String,org.springframework.batch.item.ExecutionContext>	partition(int gridSize)
void	setConfiguration(org.apache.hadoop.conf.Configuration configuration) Sets the hadoop configuration.
void	setKeyFileName(java.lang.String keyFileName) The name of the key for the file name in each ExecutionContext.
void	setKeySplitLength(java.lang.String keySplitLength) The name of the key for the file split length in each ExecutionContext.
void	setKeySplitLocations(java.lang.String keySplitLocations) The name of the key for the split locations in each ExecutionContext.
void	setKeySplitStart(java.lang.String keySplitStart) The name of the key for the file split start in each ExecutionContext.
void	setPartitionBaseIdentifier(java.lang.String keyPartition) The name of the base key for identifying partition's ExecutionContext.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractPartitioner

public AbstractPartitioner()

Method Detail

partition

public java.util.Map<java.lang.String,org.springframework.batch.item.ExecutionContext> partition(int gridSize)

Specified by:

partition in interface org.springframework.batch.core.partition.support.Partitioner

setPartitionBaseIdentifier

public void setPartitionBaseIdentifier(java.lang.String keyPartition)

The name of the base key for identifying partition's ExecutionContext. Defaults to "partition".

Parameters:

keyPartition - the value of the key

getPartitionBaseIdentifier

public java.lang.String getPartitionBaseIdentifier()

Gets the partition base identifier.

Returns:

the partition base identifier

setKeyFileName

public void setKeyFileName(java.lang.String keyFileName)

The name of the key for the file name in each ExecutionContext. Defaults to "fileName".

Parameters:

keyFileName - the value of the key

getKeyFileName

public java.lang.String getKeyFileName()

Gets the key for a file name.

Returns:

the key for a file name

setKeySplitStart

public void setKeySplitStart(java.lang.String keySplitStart)

The name of the key for the file split start in each ExecutionContext. Defaults to "splitStart".

Parameters:

keySplitStart - the value of the key

getKeySplitStart

public java.lang.String getKeySplitStart()

Gets the key for split start.

Returns:

the key for split start

setKeySplitLength

public void setKeySplitLength(java.lang.String keySplitLength)

The name of the key for the file split length in each ExecutionContext. Defaults to "splitLength".

Parameters:

keySplitLength - the value of the key

getKeySplitLength

public java.lang.String getKeySplitLength()

Gets the key for split length.

Returns:

the key for split length

setKeySplitLocations

public void setKeySplitLocations(java.lang.String keySplitLocations)

The name of the key for the split locations in each ExecutionContext. Defaults to "splitLocations".

Parameters:

keySplitLocations - the value of the key

getKeySplitLocations

public java.lang.String getKeySplitLocations()

Gets the key for split locations.

Returns:

the key for split locations

setConfiguration

@Autowired(required=false)
public void setConfiguration(org.apache.hadoop.conf.Configuration configuration)

Sets the hadoop configuration.

Parameters:

configuration - the new hadoop configuration

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

Gets the hadoop configuration.

Returns:

the hadoop configuration

createPartitions

protected abstract java.util.Map<java.lang.String,org.springframework.batch.item.ExecutionContext> createPartitions()

Creates the ExecutionContexts for partitions.

Returns:

the ExecutionContexts for partitions

createExecutionContext

protected org.springframework.batch.item.ExecutionContext createExecutionContext(org.springframework.core.io.Resource resource,
                                                                     Split split)
                                                                          throws java.io.IOException

Creates the execution context.

Parameters:

resource - the resource

split - the split

Returns:

the execution context

Throws:

java.io.IOException - Signals that an I/O exception has occurred.

createExecutionContext

protected org.springframework.batch.item.ExecutionContext createExecutionContext(org.apache.hadoop.fs.Path path,
                                                                     Split split)
                                                                          throws java.io.IOException

Creates the execution context.

Parameters:

path - the path

split - the split

Returns:

the execution context

Throws:

java.io.IOException - Signals that an I/O exception has occurred.