org.springframework.yarn.fs

Interface SmartResourceLocalizer

All Superinterfaces:

ResourceLocalizer

All Known Implementing Classes:

AbstractResourceLocalizer, DefaultResourceLocalizer

public interface SmartResourceLocalizer
extends ResourceLocalizer

SmartResourceLocalizer provides additional functionality on top of ResourceLocalizer order to handle more fine grained handling of resource localizing.

These smart functionalities includes of controlling localizing process manually, copy files into file system prior the localization and use staging concept of application files.

Logic of having methods for manually resolving, distributing and copying resources is to have an early possibility to fail fast if something goes wrong. Although one need to remember that on all circumstances calling ResourceLocalizer.getResources() should always be enough.

Author:

Janne Valkealahti

See Also:

ResourceLocalizer

Method Summary

Methods

Modifier and Type	Method and Description
boolean	clean() Cleans all leftovers what has been created during the distribute process.
void	copy() If resource localizer has knowledge to copy files into file system, calling this method effectively should do all tasks for that specific functionality.
void	distribute() If underlying implementation needs to do operations on file system or any other preparation work, calling of this method should make implementation ready to return resources from ResourceLocalizer.getResources() command.
void	resolve() Manually resolve resources before distribution process is initiated.
void	setStagingDirectory(org.apache.hadoop.fs.Path stagingDirectory) Sets the staging directory.
void	setStagingId(java.lang.String stagingId) Sets the staging id.

Methods inherited from interface org.springframework.yarn.fs.ResourceLocalizer

getResources

Method Detail

distribute

void distribute()

If underlying implementation needs to do operations on file system or any other preparation work, calling of this method should make implementation ready to return resources from ResourceLocalizer.getResources() command.

Effectively result of calling this method should be same as calling both copy() and resolve() methods manually.

resolve

void resolve()

Manually resolve resources before distribution process is initiated. Using this method should always be optional and executed before distribution if it has not already been done.

copy

void copy()

If resource localizer has knowledge to copy files into file system, calling this method effectively should do all tasks for that specific functionality.

clean

boolean clean()

Cleans all leftovers what has been created during the distribute process. These can i.e. be files in staging directory.

Returns:

true, if cleanup were attempted and was successful

setStagingDirectory

void setStagingDirectory(org.apache.hadoop.fs.Path stagingDirectory)

Sets the staging directory. If not set, path is determined by the implementation. Default value can i.e. be user's hadoop home directory.

Parameters:

stagingDirectory - the new staging directory

setStagingId

void setStagingId(java.lang.String stagingId)

Sets the staging id. Id is used together with Path set in setStagingDirectory(Path) to post fix unique runtime staging path. If not set simultaneous instances of same application may override files.

Parameters:

stagingId - the new staging id