org.springframework.data.hadoop.fs
Class HdfsResourceLoader

java.lang.Object
  org.springframework.data.hadoop.fs.HdfsResourceLoader

All Implemented Interfaces:

Closeable, DisposableBean, ResourceLoader, ResourcePatternResolver, Ordered, PriorityOrdered

public class HdfsResourceLoader
extends Object
implements ResourcePatternResolver, PriorityOrdered, Closeable, DisposableBean

Spring ResourceLoader over Hadoop FileSystem.

Author:

Costin Leau

Field Summary

Fields inherited from interface org.springframework.core.io.support.ResourcePatternResolver

CLASSPATH_ALL_URL_PREFIX

Fields inherited from interface org.springframework.core.io.ResourceLoader

CLASSPATH_URL_PREFIX

Fields inherited from interface org.springframework.core.Ordered

HIGHEST_PRECEDENCE, LOWEST_PRECEDENCE

Constructor Summary

HdfsResourceLoader(Configuration config)
Constructs a new HdfsResourceLoader instance.

HdfsResourceLoader(Configuration config, URI uri)
Constructs a new HdfsResourceLoader instance.

HdfsResourceLoader(Configuration config, URI uri, String user)
Constructs a new HdfsResourceLoader instance.

HdfsResourceLoader(FileSystem fs)
Constructs a new HdfsResourceLoader instance.

Method Summary

void	close()
void	destroy()
protected String	determineRootDir(String location)
protected Resource[]	findPathMatchingResources(String locationPattern)
ClassLoader	getClassLoader()
FileSystem	getFileSystem() Returns the Hadoop file system used by this resource loader.
int	getOrder()
Resource	getResource(String location)
Resource[]	getResources(String locationPattern)
void	setUseCodecs(boolean useCodecs) Indicates whether to use (or not) the codecs found inside the Hadoop configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsResourceLoader

public HdfsResourceLoader(Configuration config)

Constructs a new HdfsResourceLoader instance.

Parameters:

config - Hadoop configuration to use.

HdfsResourceLoader

public HdfsResourceLoader(Configuration config,
                          URI uri,
                          String user)

Constructs a new HdfsResourceLoader instance.

Parameters:

config - Hadoop configuration to use.

uri - Hadoop file system URI.

user - Hadoop user for accessing the file system.

HdfsResourceLoader

public HdfsResourceLoader(Configuration config,
                          URI uri)

Constructs a new HdfsResourceLoader instance.

Parameters:

config - Hadoop configuration to use.

uri - Hadoop file system URI.

HdfsResourceLoader

public HdfsResourceLoader(FileSystem fs)

Constructs a new HdfsResourceLoader instance.

Parameters:

fs - Hadoop file system to use.

Method Detail

getFileSystem

public FileSystem getFileSystem()

Returns the Hadoop file system used by this resource loader.

Returns:

the Hadoop file system in use.

getClassLoader

public ClassLoader getClassLoader()

Specified by:

getClassLoader in interface ResourceLoader

getResource

public Resource getResource(String location)

Specified by:

getResource in interface ResourceLoader

getResources

public Resource[] getResources(String locationPattern)
                        throws IOException

Specified by:

getResources in interface ResourcePatternResolver

Throws:

IOException

findPathMatchingResources

protected Resource[] findPathMatchingResources(String locationPattern)
                                        throws IOException

Throws:

IOException

determineRootDir

protected String determineRootDir(String location)

getOrder

public int getOrder()

Specified by:

getOrder in interface Ordered

destroy

public void destroy()
             throws IOException

Specified by:

destroy in interface DisposableBean

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Throws:

IOException

setUseCodecs

public void setUseCodecs(boolean useCodecs)

Indicates whether to use (or not) the codecs found inside the Hadoop configuration. This affects the content of the streams backing this resource - whether the raw content is delivered as is or decompressed on the fly (if the configuration allows it so). The latter is the default.

Parameters:

useCodecs - whether to use any codecs defined in the Hadoop configuration