org.springframework.yarn.fs
Interface ResourceLocalizer

All Known Implementing Classes:

DefaultResourceLocalizer

public interface ResourceLocalizer

Interface for resource localizer implementation. This loosely follows requirements for Yarn's file distribution of LocalResource instances.

Author:

Janne Valkealahti

Method Summary

boolean	clean() Cleans all leftovers what has been created during the distribute process.
void	copy() Only copy files into hdfs.
void	distribute() If underlying implementation needs to do operations on hdfs filesystem or any other preparation work, calling of this method should make implementation ready to return resources from getResources() command.
java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource>	getResources() Gets a map of LocalResource instances.
void	resolve() Only resolve localized resources.
void	setStagingDirectory(org.apache.hadoop.fs.Path stagingDirectory) Sets the staging directory.
void	setStagingId(java.lang.String stagingId) Sets the staging id.

Method Detail

getResources

java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource> getResources()

Gets a map of LocalResource instances. Underlying instances of LocalResources needs to be fully initialised including resource size and timestamp.

Returns:

The map containing LocalResource instances

distribute

void distribute()

If underlying implementation needs to do operations on hdfs filesystem or any other preparation work, calling of this method should make implementation ready to return resources from getResources() command.

Effectively result of calling this method should be same as calling both copy() and resolve() methods manually.

copy

void copy()

Only copy files into hdfs.

resolve

void resolve()

Only resolve localized resources.

setStagingDirectory

void setStagingDirectory(org.apache.hadoop.fs.Path stagingDirectory)

Sets the staging directory. If not set, path is determined by the implementation. Default value can i.e. be user's hadoop home directory.

Parameters:

stagingDirectory - the new staging directory

setStagingId

void setStagingId(java.lang.String stagingId)

Sets the staging id. Id is used together with Path set in setStagingDirectory(Path) to post fix unique runtime staging path. If not set simultaneous instances of same application may override files.

Parameters:

stagingId - the new staging id

clean

boolean clean()

Cleans all leftovers what has been created during the distribute process. These can i.e. be files in staging directory.

Returns:

true, if successful