org.springframework.data.hadoop.store.output

Class AbstractDataStreamWriter

java.lang.Object

org.springframework.data.hadoop.store.support.LifecycleObjectSupport

org.springframework.data.hadoop.store.support.StoreObjectSupport

org.springframework.data.hadoop.store.support.OutputStoreObjectSupport

org.springframework.data.hadoop.store.output.AbstractDataStreamWriter

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.InitializingBean, org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle

Direct Known Subclasses:

OutputStreamWriter, TextFileWriter

public abstract class AbstractDataStreamWriter
extends OutputStoreObjectSupport

A AbstractDataStreamWriter is a base implementation handling streams with a raw hdfs files.

Author:

Janne Valkealahti

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_MAX_OPEN_ATTEMPTS

Constructor Summary

Constructors

Constructor and Description
AbstractDataStreamWriter(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path basePath, CodecInfo codec) Instantiates a new abstract data stream writer.

Method Summary

Methods

Modifier and Type	Method and Description
protected StreamsHolder<java.io.OutputStream>	getOutput() Gets the output.
protected long	getPosition(StreamsHolder<java.io.OutputStream> holder) Gets the current stream writing position.
void	setMaxOpenAttempts(int maxOpenAttempts) Sets the max open attempts trying to find a suitable path for output stream.

Methods inherited from class org.springframework.data.hadoop.store.support.OutputStoreObjectSupport

findInitFiles, getOutputContext, getResolvedPath, initOutputContext, isAppendable, isFileWriteable, isOverwrite, onInit, renameFile, setAppendable, setFileNamingStrategy, setInWritingPrefix, setInWritingSuffix, setOverwrite, setRolloverStrategy, setWritePosition

Methods inherited from class org.springframework.data.hadoop.store.support.StoreObjectSupport

doStart, doStop, getCodec, getConfiguration, getPath, handleIdleTimeout, isCompressed, resetIdleTimeout, setIdleTimeout

Methods inherited from class org.springframework.data.hadoop.store.support.LifecycleObjectSupport

afterPropertiesSet, getBeanFactory, getPhase, getStoreEventPublisher, getTaskExecutor, getTaskScheduler, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setStoreEventPublisher, setTaskExecutor, setTaskScheduler, start, stop, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_MAX_OPEN_ATTEMPTS

public static final int DEFAULT_MAX_OPEN_ATTEMPTS

See Also:

Constant Field Values

Constructor Detail

AbstractDataStreamWriter

public AbstractDataStreamWriter(org.apache.hadoop.conf.Configuration configuration,
                        org.apache.hadoop.fs.Path basePath,
                        CodecInfo codec)

Instantiates a new abstract data stream writer.

Parameters:

configuration - the hadoop configuration

basePath - the hdfs path

codec - the compression codec info

Method Detail

setMaxOpenAttempts

public void setMaxOpenAttempts(int maxOpenAttempts)

Sets the max open attempts trying to find a suitable path for output stream. Only positive values are allowed and any attempt to set this to less than 1 will automatically reset value to exactly 1.

Parameters:

maxOpenAttempts - the new max open attempts

getOutput

protected StreamsHolder<java.io.OutputStream> getOutput()
                                                 throws java.io.IOException

Gets the output.

Returns:

the output

Throws:

java.io.IOException - Signals that an I/O exception has occurred.

getPosition

protected long getPosition(StreamsHolder<java.io.OutputStream> holder)
                    throws java.io.IOException

Gets the current stream writing position.

Parameters:

holder - the holder for output streams

Returns:

the position

Throws:

java.io.IOException - Signals that an I/O exception has occurred.