org.springframework.yarn.config.annotation.builders

Class YarnClientBuilder

java.lang.Object

org.springframework.data.hadoop.config.common.annotation.AbstractAnnotationBuilder<O>

org.springframework.data.hadoop.config.common.annotation.AbstractConfiguredAnnotationBuilder<YarnClient,YarnClientConfigurer,YarnClientBuilder>

org.springframework.yarn.config.annotation.builders.YarnClientBuilder

All Implemented Interfaces:

AnnotationBuilder<YarnClient>, YarnClientConfigurer

public class YarnClientBuilder
extends AbstractConfiguredAnnotationBuilder<YarnClient,YarnClientConfigurer,YarnClientBuilder>
implements YarnClientConfigurer

AnnotationBuilder for YarnClient.

Author:

Janne Valkealahti

Constructor Summary

Constructors

Constructor and Description
YarnClientBuilder() Instantiates a new yarn client builder.

Method Summary

Methods

Modifier and Type	Method and Description
YarnClientConfigurer	appName(java.lang.String appName) Specify a yarn application name.
YarnClientConfigurer	appType(java.lang.String appType) Specify a yarn application type.
YarnClientConfigurer	clientClass(java.lang.Class<? extends YarnClient> clazz) Specify a YarnClient class.
YarnClientConfigurer	clientClass(java.lang.String clazz) Specify a YarnClient as a fully qualified class name.
void	configuration(org.apache.hadoop.conf.Configuration configuration)
YarnClientConfigurer	labelExpression(java.lang.String labelExpression) Specify a yarn application submission label expression.
YarnClientConfigurer	masterCommands(java.lang.String... commands) Specify a raw array of commands used to start an application master.
YarnClientConfigurer	memory(int memory) Specify a yarn application containers memory reservation.
YarnClientConfigurer	memory(java.lang.String memory) Specify a yarn application containers memory reservation.
protected YarnClient	performBuild() Subclasses must implement this method to build the object that is being returned.
YarnClientConfigurer	priority(java.lang.Integer priority) Specify a yarn application priority.
YarnClientConfigurer	queue(java.lang.String queue) Specify a yarn application submission queue.
void	setCommands(java.lang.String... commands)
void	setEnvironment(java.util.Map<java.lang.String,java.lang.String> environment)
void	setResourceLocalizer(ResourceLocalizer resourceLocalizer)
YarnClientConfigurer	virtualCores(java.lang.Integer virtualCores) Specify a yarn application virtual core resource count.
DefaultClientMasterRunnerConfigurer	withMasterRunner() Add commands for starting YarnAppmaster.

Methods inherited from class org.springframework.data.hadoop.config.common.annotation.AbstractConfiguredAnnotationBuilder

apply, apply, beforeConfigureMains, beforeConfigurePosts, beforeInit, doBuild, getConfigurer, getConfigurers, getOrApply, getOrBuild, getSharedObject, getSharedObjects, objectPostProcessor, postProcess, removeConfigurer, removeConfigurers, setSharedObject

Methods inherited from class org.springframework.data.hadoop.config.common.annotation.AbstractAnnotationBuilder

build, getObject

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnClientBuilder

public YarnClientBuilder()

Instantiates a new yarn client builder.

Method Detail

performBuild

protected YarnClient performBuild()
                           throws java.lang.Exception

Description copied from class: AbstractConfiguredAnnotationBuilder

Subclasses must implement this method to build the object that is being returned.

Specified by:

performBuild in class AbstractConfiguredAnnotationBuilder<YarnClient,YarnClientConfigurer,YarnClientBuilder>

Returns:

Object build by this builder

Throws:

java.lang.Exception - on error

withMasterRunner

public DefaultClientMasterRunnerConfigurer withMasterRunner()
                                                     throws java.lang.Exception

Add commands for starting YarnAppmaster.

Specified by:

withMasterRunner in interface YarnClientConfigurer

Returns:

the client master runner configurer

Throws:

java.lang.Exception - the exception

appName

public YarnClientConfigurer appName(java.lang.String appName)

Description copied from interface: YarnClientConfigurer

Specify a yarn application name.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .appName("myAppName");
 }
 

XML:

 <yarn:client app-name="myAppName"/>
 

Specified by:

appName in interface YarnClientConfigurer

Parameters:

appName - The Yarn application name

Returns:

YarnClientConfigurer for chaining

appType

public YarnClientConfigurer appType(java.lang.String appType)

Description copied from interface: YarnClientConfigurer

Specify a yarn application type. Type is a simple string user will see as a field when querying applications from a resource manager. For example, MapReduce jobs are using type MAPREDUCE and other applications defaults to YARN.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .appType("BOOT");
 }
 

XML:
No equivalent

Specified by:

appType in interface YarnClientConfigurer

Parameters:

appType - The Yarn application type

Returns:

YarnClientConfigurer for chaining

masterCommands

public YarnClientConfigurer masterCommands(java.lang.String... commands)

Description copied from interface: YarnClientConfigurer

Specify a raw array of commands used to start an application master.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .masterCommands("java -jar MyApp.jar", "1><LOG_DIR>/Appmaster.stdout", "2><LOG_DIR>/Appmaster.stderr");
 }
 

XML:

 <yarn:client>
   <yarn:master-command>
     <![CDATA[
     java -jar MyApp.jar
     1><LOG_DIR>/Appmaster.stdout
     2><LOG_DIR>/Appmaster.stderr
     ]]>
   </yarn:master-command>
 </yarn:client>
 

Specified by:

masterCommands in interface YarnClientConfigurer

Parameters:

commands - The Yarn container commands

Returns:

YarnAppmasterConfigurer for chaining

priority

public YarnClientConfigurer priority(java.lang.Integer priority)

Description copied from interface: YarnClientConfigurer

Specify a yarn application priority.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .priority(0);
 }
 

XML:

 <yarn:client priority="0"/>
 

Specified by:

priority in interface YarnClientConfigurer

Parameters:

priority - The Yarn application priority

Returns:

YarnClientConfigurer for chaining

queue

public YarnClientConfigurer queue(java.lang.String queue)

Description copied from interface: YarnClientConfigurer

Specify a yarn application submission queue. Specified queue is a one client requests but it's not necessarily a one where application is placed. Some Yarn schedulers may choose to change this so user should be aware of how Yarn is setup.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .queue("default");
 }
 

XML:

 <yarn:client queue="default"/>
 

Specified by:

queue in interface YarnClientConfigurer

Parameters:

queue - The Yarn application submission queue

Returns:

YarnClientConfigurer for chaining

labelExpression

public YarnClientConfigurer labelExpression(java.lang.String labelExpression)

Description copied from interface: YarnClientConfigurer

Specify a yarn application submission label expression. If this is set, all containers of this application without setting label expression in resource request will get allocated resources on only those nodes that satisfy this label expression. If different label expression of this app and resource request are set at the same time, the one set in resource request will be used when allocating container.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .labelExpression("expression");
 }
 

XML:
No equivalent

Specified by:

labelExpression in interface YarnClientConfigurer

Parameters:

labelExpression - The Yarn application label expression

Returns:

YarnClientConfigurer for chaining

memory

public YarnClientConfigurer memory(int memory)

Description copied from interface: YarnClientConfigurer

Specify a yarn application containers memory reservation. The memory argument is given as MegaBytes.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .memory(1024);
 }
 

XML:

 <yarn:client memory="1024"/>
 

Specified by:

memory in interface YarnClientConfigurer

Parameters:

memory - The Yarn application containers memory reservation

Returns:

YarnClientConfigurer for chaining

See Also:

YarnClientConfigurer.memory(String)

memory

public YarnClientConfigurer memory(java.lang.String memory)

Description copied from interface: YarnClientConfigurer

Specify a yarn application containers memory reservation. The memory argument is given as MegaBytes if value is a plain number. Shortcuts like 1G and 500M can be used which translates to 1024 and 500 respectively.

This method is equivalent to #memory(int) so that argument can be given as a String.

NOTE: be careful not to use a too low settings like 1000K or 1000B because those are rounded down to full MBs and thus becomes a zero. Also too high values may make resource allocation to behave badly.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .memory("1G");
 }
 

XML:

 <yarn:client memory="1024"/>
 

Specified by:

memory in interface YarnClientConfigurer

Parameters:

memory - The Yarn application containers memory reservation

Returns:

YarnClientConfigurer for chaining

virtualCores

public YarnClientConfigurer virtualCores(java.lang.Integer virtualCores)

Description copied from interface: YarnClientConfigurer

Specify a yarn application virtual core resource count.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .virtualCores(1);
 }
 

XML:

 <yarn:client virtualcores="1"/>
 

Specified by:

virtualCores in interface YarnClientConfigurer

Parameters:

virtualCores - The Yarn application virtual core resource count

Returns:

YarnClientConfigurer for chaining

clientClass

public YarnClientConfigurer clientClass(java.lang.Class<? extends YarnClient> clazz)

Description copied from interface: YarnClientConfigurer

Specify a YarnClient class.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .clientClass(MyYarnClient.class);
 }
 

XML:
No equivalent

Specified by:

clientClass in interface YarnClientConfigurer

Parameters:

clazz - The Yarn client class

Returns:

YarnClientConfigurer for chaining

clientClass

public YarnClientConfigurer clientClass(java.lang.String clazz)

Description copied from interface: YarnClientConfigurer

Specify a YarnClient as a fully qualified class name.

JavaConfig:

 public void configure(YarnClientConfigure client) throws Exception {
   client
     .clientClass("com.example.MyYarnClient");
 }
 

XML:
No equivalent

Specified by:

clientClass in interface YarnClientConfigurer

Parameters:

clazz - The Yarn client class

Returns:

YarnClientConfigurer for chaining

configuration

public void configuration(org.apache.hadoop.conf.Configuration configuration)

setCommands

public void setCommands(java.lang.String... commands)

setResourceLocalizer

public void setResourceLocalizer(ResourceLocalizer resourceLocalizer)

setEnvironment

public void setEnvironment(java.util.Map<java.lang.String,java.lang.String> environment)