Spring for Apache Hadoop

org.springframework.yarn.config.annotation.builders
Interface YarnEnvironmentConfigurer

All Known Implementing Classes:
YarnEnvironmentBuilder

public interface YarnEnvironmentConfigurer

Interface for YarnEnvironmentBuilder used from a SpringYarnConfigurerAdapter.

Typically configuration is used as shown below.

 @Configuration
 @EnableYarn
 static class Config extends SpringYarnConfigurerAdapter {

   @Override
   public void configure(YarnEnvironmentBuilder environment) throws Exception {
     environment
       .withClasspath()
         .entry("cpEntry1")
         .entry("cpEntry2")
         .useDefaultYarnClasspath(true);
   }

 }
 

Author:
Janne Valkealahti

Method Summary
 YarnEnvironmentConfigurer entry(java.lang.String key, java.lang.String value)
          Specify an environment variable.
 YarnEnvironmentConfigurer includeLocalSystemEnv(boolean includeLocalSystemEnv)
          Specify if existing system environment variables should be included automatically.
 YarnEnvironmentConfigurer propertiesLocation(java.lang.String... locations)
          Specify properties locations.
 EnvironmentClasspathConfigurer withClasspath()
          Specify a classpath environment variable.
 PropertiesConfigurer<YarnEnvironmentConfigurer> withProperties()
          Specify properties with a PropertiesConfigurer.
 

Method Detail

withClasspath

EnvironmentClasspathConfigurer withClasspath()
                                             throws java.lang.Exception
Specify a classpath environment variable.

Applies a new DefaultEnvironmentClasspathConfigurer into current builder. Equivalents between JavaConfig and XML are shown below.

JavaConfig:

 public void configure(YarnEnvironmentBuilder environment) throws Exception {
   environment
     .withClasspath()
       .entry("cpEntry1")
       .entry("cpEntry2")
       .useDefaultYarnClasspath(true);
 }
 

XML:

 <yarn:environment>
   <yarn:classpath use-default-yarn-classpath="true" delimiter=":">
     cpEntry1
     cpEntry2
   </yarn:classpath>
 </yarn:environment>
 

Returns:
DefaultEnvironmentClasspathConfigurer for classpath
Throws:
java.lang.Exception - if error occurred

entry

YarnEnvironmentConfigurer entry(java.lang.String key,
                                java.lang.String value)
Specify an environment variable.

JavaConfig:

 public void configure(YarnEnvironmentConfigure environment) throws Exception {
   environment
     .entry("myKey1","myValue1")
     .entry("myKey2","myValue2");
 }
 

XML:

 <yarn:environment>
   myKey1=myValue1
   myKey2=myValue2
 </yarn:environment>
 

Parameters:
key - The environment key
value - The environment value
Returns:
YarnEnvironmentConfigurer for chaining

propertiesLocation

YarnEnvironmentConfigurer propertiesLocation(java.lang.String... locations)
                                             throws java.io.IOException
Specify properties locations.

JavaConfig:

 public void configure(YarnEnvironmentConfigure environment) throws Exception {
   environment
     .entry("myKey1","myValue1")
     .entry("myKey2","myValue2")
     .propertiesLocation("cfg-1.properties", "cfg-2.properties");
 }
 

XML:

 <yarn:environment properties-location="cfg-1.properties, cfg-2.properties">
   myKey1=myValue1
   myKey2=myValue2
 </yarn:environment>
 

Parameters:
locations - The properties file locations
Returns:
YarnEnvironmentConfigurer for chaining
Throws:
java.io.IOException - if error occurred

includeLocalSystemEnv

YarnEnvironmentConfigurer includeLocalSystemEnv(boolean includeLocalSystemEnv)
Specify if existing system environment variables should be included automatically.

JavaConfig:

 public void configure(YarnEnvironmentConfigure environment) throws Exception {
   environment
     .includeLocalSystemEnv(false);
 }
 

XML:

 <yarn:environment include-local-system-env="false"/>
 

Parameters:
includeLocalSystemEnv - if system env variables should be included
Returns:
YarnEnvironmentConfigurer for chaining

withProperties

PropertiesConfigurer<YarnEnvironmentConfigurer> withProperties()
                                                               throws java.lang.Exception
Specify properties with a PropertiesConfigurer.

JavaConfig:

 public void configure(YarnEnvironmentConfigure environment) throws Exception {
   Properties props = new Properties();
   environment
     .withProperties()
       .properties(props)
       .property("myKey1", ",myValue1")
       .and();
 }
 

XML:

 <util:properties id="props" location="props.properties"/>
   myValue1
 </util:properties>
 <yarn:environment properties-ref="props"/>
 

Returns:
PropertiesConfigurer for chaining
Throws:
java.lang.Exception - if error occurred

Spring for Apache Hadoop