Spring for Apache Hadoop

org.springframework.data.hadoop.fs
Class HdfsResourceLoader

java.lang.Object
  extended by org.springframework.data.hadoop.fs.HdfsResourceLoader
All Implemented Interfaces:
java.io.Closeable, org.springframework.beans.factory.DisposableBean, org.springframework.core.io.ResourceLoader, org.springframework.core.io.support.ResourcePatternResolver, org.springframework.core.Ordered, org.springframework.core.PriorityOrdered

public class HdfsResourceLoader
extends java.lang.Object
implements org.springframework.core.io.support.ResourcePatternResolver, org.springframework.core.PriorityOrdered, java.io.Closeable, org.springframework.beans.factory.DisposableBean

Spring ResourceLoader over Hadoop FileSystem.

Author:
Costin Leau

Field Summary
 
Fields inherited from interface org.springframework.core.io.support.ResourcePatternResolver
CLASSPATH_ALL_URL_PREFIX
 
Fields inherited from interface org.springframework.core.io.ResourceLoader
CLASSPATH_URL_PREFIX
 
Fields inherited from interface org.springframework.core.Ordered
HIGHEST_PRECEDENCE, LOWEST_PRECEDENCE
 
Constructor Summary
HdfsResourceLoader(org.apache.hadoop.conf.Configuration config)
          Constructs a new HdfsResourceLoader instance.
HdfsResourceLoader(org.apache.hadoop.conf.Configuration config, java.net.URI uri)
          Constructs a new HdfsResourceLoader instance.
HdfsResourceLoader(org.apache.hadoop.conf.Configuration config, java.net.URI uri, java.lang.String user)
          Constructs a new HdfsResourceLoader instance.
HdfsResourceLoader(org.apache.hadoop.fs.FileSystem fs)
          Constructs a new HdfsResourceLoader instance.
 
Method Summary
 void close()
           
 void destroy()
           
protected  java.lang.String determineRootDir(java.lang.String location)
           
protected  org.springframework.core.io.Resource[] findPathMatchingResources(java.lang.String locationPattern)
           
 java.lang.ClassLoader getClassLoader()
           
 org.apache.hadoop.fs.FileSystem getFileSystem()
          Returns the Hadoop file system used by this resource loader.
 int getOrder()
           
 org.springframework.core.io.Resource getResource(java.lang.String location)
           
 org.springframework.core.io.Resource[] getResources(java.lang.String locationPattern)
           
 void setUseCodecs(boolean useCodecs)
          Indicates whether to use (or not) the codecs found inside the Hadoop configuration.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

HdfsResourceLoader

public HdfsResourceLoader(org.apache.hadoop.conf.Configuration config)
Constructs a new HdfsResourceLoader instance.

Parameters:
config - Hadoop configuration to use.

HdfsResourceLoader

public HdfsResourceLoader(org.apache.hadoop.conf.Configuration config,
                          java.net.URI uri,
                          java.lang.String user)
Constructs a new HdfsResourceLoader instance.

Parameters:
config - Hadoop configuration to use.
uri - Hadoop file system URI.
user - Hadoop user for accessing the file system.

HdfsResourceLoader

public HdfsResourceLoader(org.apache.hadoop.conf.Configuration config,
                          java.net.URI uri)
Constructs a new HdfsResourceLoader instance.

Parameters:
config - Hadoop configuration to use.
uri - Hadoop file system URI.

HdfsResourceLoader

public HdfsResourceLoader(org.apache.hadoop.fs.FileSystem fs)
Constructs a new HdfsResourceLoader instance.

Parameters:
fs - Hadoop file system to use.
Method Detail

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()
Returns the Hadoop file system used by this resource loader.

Returns:
the Hadoop file system in use.

getClassLoader

public java.lang.ClassLoader getClassLoader()
Specified by:
getClassLoader in interface org.springframework.core.io.ResourceLoader

getResource

public org.springframework.core.io.Resource getResource(java.lang.String location)
Specified by:
getResource in interface org.springframework.core.io.ResourceLoader

getResources

public org.springframework.core.io.Resource[] getResources(java.lang.String locationPattern)
                                                    throws java.io.IOException
Specified by:
getResources in interface org.springframework.core.io.support.ResourcePatternResolver
Throws:
java.io.IOException

findPathMatchingResources

protected org.springframework.core.io.Resource[] findPathMatchingResources(java.lang.String locationPattern)
                                                                    throws java.io.IOException
Throws:
java.io.IOException

determineRootDir

protected java.lang.String determineRootDir(java.lang.String location)

getOrder

public int getOrder()
Specified by:
getOrder in interface org.springframework.core.Ordered

destroy

public void destroy()
             throws java.io.IOException
Specified by:
destroy in interface org.springframework.beans.factory.DisposableBean
Throws:
java.io.IOException

close

public void close()
           throws java.io.IOException
Specified by:
close in interface java.io.Closeable
Throws:
java.io.IOException

setUseCodecs

public void setUseCodecs(boolean useCodecs)
Indicates whether to use (or not) the codecs found inside the Hadoop configuration. This affects the content of the streams backing this resource - whether the raw content is delivered as is or decompressed on the fly (if the configuration allows it so). The latter is the default.

Parameters:
useCodecs - whether to use any codecs defined in the Hadoop configuration

Spring for Apache Hadoop