org.springframework.data.hadoop.store.output

Class PartitionTextFileWriter<K>

java.lang.Object

org.springframework.data.hadoop.store.support.LifecycleObjectSupport

org.springframework.data.hadoop.store.output.AbstractPartitionDataStoreWriter<java.lang.String,K>

org.springframework.data.hadoop.store.output.PartitionTextFileWriter<K>

Type Parameters:

K - the type of a partition key

All Implemented Interfaces:

java.io.Closeable, java.io.Flushable, java.lang.AutoCloseable, org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.InitializingBean, org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle, DataStoreWriter<java.lang.String>, DataWriter<java.lang.String>, PartitionDataStoreWriter<java.lang.String,K>

public class PartitionTextFileWriter<K>
extends AbstractPartitionDataStoreWriter<java.lang.String,K>

Implementation of a PartitionDataStoreWriter writing text data using a partitioning.

Author:

Janne Valkealahti, Rodrigo Meneses

Constructor Summary

Constructors

Constructor and Description
PartitionTextFileWriter(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path basePath, CodecInfo codec, PartitionStrategy<java.lang.String,K> partitionStrategy) Instantiates a new text file partitioned writer.

Method Summary

Methods

Modifier and Type	Method and Description
protected DataStoreWriter<java.lang.String>	createWriter(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path path, CodecInfo codec) Need to be implemented by a subclass for an actual writer.

Methods inherited from class org.springframework.data.hadoop.store.output.AbstractPartitionDataStoreWriter

close, destroyWriter, doStart, doStop, flush, getBasePath, getCodec, getConfiguration, getFileNamingStrategyFactory, getIdleTimeout, getInWritingPrefix, getInWritingSuffix, getMaxOpenAttempts, getRolloverStrategyFactory, isAppendable, isClosed, isOverwrite, onInit, setAppendable, setFileNamingStrategyFactory, setIdleTimeout, setInWritingPrefix, setInWritingSuffix, setMaxOpenAttempts, setOverwrite, setRolloverStrategyFactory, write, write

Methods inherited from class org.springframework.data.hadoop.store.support.LifecycleObjectSupport

afterPropertiesSet, getBeanFactory, getPhase, getStoreEventPublisher, getTaskExecutor, getTaskScheduler, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setStoreEventPublisher, setTaskExecutor, setTaskScheduler, start, stop, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionTextFileWriter

public PartitionTextFileWriter(org.apache.hadoop.conf.Configuration configuration,
                       org.apache.hadoop.fs.Path basePath,
                       CodecInfo codec,
                       PartitionStrategy<java.lang.String,K> partitionStrategy)

Instantiates a new text file partitioned writer.

Parameters:

configuration - the hadoop configuration

basePath - the hdfs path

codec - the compression codec info

partitionStrategy - the partition strategy

Method Detail

createWriter

protected DataStoreWriter<java.lang.String> createWriter(org.apache.hadoop.conf.Configuration configuration,
                                             org.apache.hadoop.fs.Path path,
                                             CodecInfo codec)

Description copied from class: AbstractPartitionDataStoreWriter

Need to be implemented by a subclass for an actual writer.

Specified by:

createWriter in class AbstractPartitionDataStoreWriter<java.lang.String,K>

Parameters:

configuration - the configuration

path - the base path

codec - the codec

Returns:

the data store writer