org.springframework.data.hadoop.store.dataset

Class ParquetDatasetStoreWriter<T>

java.lang.Object

org.springframework.data.hadoop.store.support.LifecycleObjectSupport

org.springframework.data.hadoop.store.dataset.DatasetStoreObjectSupport

org.springframework.data.hadoop.store.dataset.AbstractDatasetStoreWriter<T,org.apache.avro.generic.GenericRecord>

org.springframework.data.hadoop.store.dataset.ParquetDatasetStoreWriter<T>

Type Parameters:

T - the type of entity to write

All Implemented Interfaces:

java.io.Closeable, java.io.Flushable, java.lang.AutoCloseable, org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.InitializingBean, org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle, DataStoreWriter<T>, DataWriter<T>

public class ParquetDatasetStoreWriter<T>
extends AbstractDatasetStoreWriter<T,org.apache.avro.generic.GenericRecord>

A DataStoreWriter for writing Datasets using the Parquet format.

Author:

Thomas Risberg, Janne Valkealahti

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.avro.Schema	schema

Constructor Summary

Constructors

Constructor and Description
ParquetDatasetStoreWriter(java.lang.Class<T> entityClass, DatasetRepositoryFactory datasetRepositoryFactory) Instantiates a new DataStoreWriter for writing Parquet records to a org.kitesdk.data.Dataset.
ParquetDatasetStoreWriter(java.lang.Class<T> entityClass, DatasetRepositoryFactory datasetRepositoryFactory, DatasetDefinition datasetDefinition) Instantiates a new DataStoreWriter for writing Parquet records to a org.kitesdk.data.Dataset.

Method Summary

Methods

Modifier and Type	Method and Description
protected org.apache.avro.generic.GenericRecord	convertEntity(T entity) Convert entity to be written into a entity used by a writer.
protected org.kitesdk.data.DatasetWriter<org.apache.avro.generic.GenericRecord>	createWriter() Creates a DatasetWriter.
void	write(T entity) Write an entity.

Methods inherited from class org.springframework.data.hadoop.store.dataset.AbstractDatasetStoreWriter

close, flush, getDatasetDefinition, getDatasetRepositoryFactory, getEntityClass, handleTimeout

Methods inherited from class org.springframework.data.hadoop.store.dataset.DatasetStoreObjectSupport

doStart, doStop, onInit, resetIdleTimeout, setCloseTimeout, setIdleTimeout

Methods inherited from class org.springframework.data.hadoop.store.support.LifecycleObjectSupport

afterPropertiesSet, getBeanFactory, getPhase, getStoreEventPublisher, getTaskExecutor, getTaskScheduler, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setStoreEventPublisher, setTaskExecutor, setTaskScheduler, start, stop, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

schema

protected volatile org.apache.avro.Schema schema

Constructor Detail

ParquetDatasetStoreWriter

public ParquetDatasetStoreWriter(java.lang.Class<T> entityClass,
                         DatasetRepositoryFactory datasetRepositoryFactory)

Instantiates a new DataStoreWriter for writing Parquet records to a org.kitesdk.data.Dataset.

Parameters:

entityClass - the Class that the writer will write to the Dataset

datasetRepositoryFactory - the DatasetRepositoryFactory to be used for the writer

ParquetDatasetStoreWriter

public ParquetDatasetStoreWriter(java.lang.Class<T> entityClass,
                         DatasetRepositoryFactory datasetRepositoryFactory,
                         DatasetDefinition datasetDefinition)

Instantiates a new DataStoreWriter for writing Parquet records to a org.kitesdk.data.Dataset.

Parameters:

entityClass - the Class that the writer will write to the Dataset

datasetRepositoryFactory - the DatasetRepositoryFactory to be used for the writer

datasetDefinition - the DatasetDefinition to be used for the writer

Method Detail

write

public void write(T entity)
           throws java.io.IOException

Description copied from interface: DataWriter

Write an entity.

Specified by:

write in interface DataWriter<T>

Overrides:

write in class AbstractDatasetStoreWriter<T,org.apache.avro.generic.GenericRecord>

Parameters:

entity - the entity to write

Throws:

java.io.IOException - if an I/O error occurs

createWriter

protected org.kitesdk.data.DatasetWriter<org.apache.avro.generic.GenericRecord> createWriter()

Description copied from class: AbstractDatasetStoreWriter

Creates a DatasetWriter.

Specified by:

createWriter in class AbstractDatasetStoreWriter<T,org.apache.avro.generic.GenericRecord>

Returns:

the dataset writer

convertEntity

protected org.apache.avro.generic.GenericRecord convertEntity(T entity)

Description copied from class: AbstractDatasetStoreWriter

Convert entity to be written into a entity used by a writer.

Specified by:

convertEntity in class AbstractDatasetStoreWriter<T,org.apache.avro.generic.GenericRecord>

Parameters:

entity - the entity

Returns:

the converted entity