org.springframework.data.hadoop.batch.item

Class HdfsItemWriter<T>

java.lang.Object

org.springframework.data.hadoop.batch.item.HdfsItemWriter<T>

All Implemented Interfaces:

org.springframework.batch.item.ItemStream, org.springframework.batch.item.ItemStreamWriter<T>, org.springframework.batch.item.ItemWriter<T>

public class HdfsItemWriter<T>
extends java.lang.Object
implements org.springframework.batch.item.ItemStreamWriter<T>

An ItemWriter implementation used to write the incoming items to HDFS. Due to the HDFS limitation that files cannot be deleted or modified, the ability to roll back a file to a previously known state is not possible. This prevents the ability to restart using this ItemWriter.
This ItemWriter is not thread-safe.

Author:

Michael Minella

Constructor Summary

Constructors

Constructor and Description
HdfsItemWriter(org.apache.hadoop.fs.FileSystem fileSystem, org.springframework.core.serializer.Serializer<T> itemSerializer, java.lang.String fileName) Constructor

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
protected void	doWrite(java.util.List<? extends T> items) Performs the actual write to the store via the template.
void	open(org.springframework.batch.item.ExecutionContext executionContext)
void	update(org.springframework.batch.item.ExecutionContext executionContext)
void	write(java.util.List<? extends T> items)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsItemWriter

public HdfsItemWriter(org.apache.hadoop.fs.FileSystem fileSystem,
              org.springframework.core.serializer.Serializer<T> itemSerializer,
              java.lang.String fileName)

Constructor

Parameters:

fileSystem - - HDFS FileSystem reference

itemSerializer - - Strategy for serializing items

fileName - - Name of the file to be written to

Method Detail

doWrite

protected void doWrite(java.util.List<? extends T> items)

Performs the actual write to the store via the template. This can be overridden by a subclass if necessary.

Parameters:

items - the list of items to be persisted.

open

public void open(org.springframework.batch.item.ExecutionContext executionContext)
          throws org.springframework.batch.item.ItemStreamException

Specified by:

open in interface org.springframework.batch.item.ItemStream

Throws:

org.springframework.batch.item.ItemStreamException

update

public void update(org.springframework.batch.item.ExecutionContext executionContext)
            throws org.springframework.batch.item.ItemStreamException

Specified by:

update in interface org.springframework.batch.item.ItemStream

Throws:

org.springframework.batch.item.ItemStreamException

write

public void write(java.util.List<? extends T> items)
           throws java.lang.Exception

Specified by:

write in interface org.springframework.batch.item.ItemWriter<T>

Throws:

java.lang.Exception

close

public void close()

Specified by:

close in interface org.springframework.batch.item.ItemStream