org.springframework.data.hadoop.config.annotation.builders

Class HadoopConfigBuilder

java.lang.Object

org.springframework.data.hadoop.config.common.annotation.AbstractAnnotationBuilder<O>

org.springframework.data.hadoop.config.common.annotation.AbstractConfiguredAnnotationBuilder<org.apache.hadoop.conf.Configuration,HadoopConfigConfigurer,HadoopConfigBuilder>

org.springframework.data.hadoop.config.annotation.builders.HadoopConfigBuilder

All Implemented Interfaces:

HadoopConfigConfigurer, AnnotationBuilder<org.apache.hadoop.conf.Configuration>, PropertiesConfigurerAware, ResourceConfigurerAware, SecurityConfigurerAware

public final class HadoopConfigBuilder
extends AbstractConfiguredAnnotationBuilder<org.apache.hadoop.conf.Configuration,HadoopConfigConfigurer,HadoopConfigBuilder>
implements PropertiesConfigurerAware, ResourceConfigurerAware, SecurityConfigurerAware, HadoopConfigConfigurer

AnnotationBuilder for Configuration.

Author:

Janne Valkealahti

Constructor Summary

Constructors

Constructor and Description
HadoopConfigBuilder() Instantiates a new yarn config builder.
HadoopConfigBuilder(ObjectPostProcessor<java.lang.Object> objectPostProcessor) Instantiates a new hadoop config builder.

Method Summary

Methods

Modifier and Type	Method and Description
void	configureProperties(java.util.Properties properties) Configure Properties.
void	configureResources(java.util.Set<org.springframework.core.io.Resource> resources) Configure Resources.
void	configureSecurity(HadoopSecurity hadoopSecurity) Configure HadoopSecurity.
HadoopConfigConfigurer	fileSystemUri(java.lang.String uri) Specify a Hdfs file system uri.
java.util.Properties	getProperties() Gets the Properties.
java.util.Set<org.springframework.core.io.Resource>	getResources() Gets the Resources.
HadoopSecurity	getSecurity() Gets the HadoopSecurity.
HadoopConfigConfigurer	jobHistoryAddress(java.lang.String jobHistoryAddress) Specify a MapReduce job history address.
HadoopConfigConfigurer	loadDefaults(boolean loadDefaults) Specify if Hadoop Configuration is initially based on default values.
protected org.apache.hadoop.conf.Configuration	performBuild() Subclasses must implement this method to build the object that is being returned.
HadoopConfigConfigurer	resourceManagerAddress(java.lang.String address) Specify a Yarn resource manager address.
PropertiesConfigurer<HadoopConfigConfigurer>	withProperties() Specify configuration options as properties with a PropertiesConfigurer.
ResourceConfigurer<HadoopConfigConfigurer>	withResources() Specify configuration options as resource properties with a ResourceConfigurer.
SecurityConfigurer<HadoopConfigConfigurer>	withSecurity() Specify security options with a SecurityConfigurer.

Methods inherited from class org.springframework.data.hadoop.config.common.annotation.AbstractConfiguredAnnotationBuilder

apply, apply, beforeConfigureMains, beforeConfigurePosts, beforeInit, doBuild, getConfigurer, getConfigurers, getOrApply, getOrBuild, getSharedObject, getSharedObjects, objectPostProcessor, postProcess, removeConfigurer, removeConfigurers, setSharedObject

Methods inherited from class org.springframework.data.hadoop.config.common.annotation.AbstractAnnotationBuilder

build, getObject

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopConfigBuilder

public HadoopConfigBuilder()

Instantiates a new yarn config builder.

HadoopConfigBuilder

public HadoopConfigBuilder(ObjectPostProcessor<java.lang.Object> objectPostProcessor)

Instantiates a new hadoop config builder.

Parameters:

objectPostProcessor - the object post processor

Method Detail

performBuild

protected org.apache.hadoop.conf.Configuration performBuild()
                                                     throws java.lang.Exception

Description copied from class: AbstractConfiguredAnnotationBuilder

Subclasses must implement this method to build the object that is being returned.

Specified by:

performBuild in class AbstractConfiguredAnnotationBuilder<org.apache.hadoop.conf.Configuration,HadoopConfigConfigurer,HadoopConfigBuilder>

Returns:

Object build by this builder

Throws:

java.lang.Exception - on error

configureProperties

public void configureProperties(java.util.Properties properties)

Description copied from interface: PropertiesConfigurerAware

Configure Properties.

Specified by:

configureProperties in interface PropertiesConfigurerAware

Parameters:

properties - the properties

configureResources

public void configureResources(java.util.Set<org.springframework.core.io.Resource> resources)

Description copied from interface: ResourceConfigurerAware

Configure Resources.

Specified by:

configureResources in interface ResourceConfigurerAware

Parameters:

resources - the resources

configureSecurity

public void configureSecurity(HadoopSecurity hadoopSecurity)

Description copied from interface: SecurityConfigurerAware

Configure HadoopSecurity.

Specified by:

configureSecurity in interface SecurityConfigurerAware

Parameters:

hadoopSecurity - the hadoop security

withResources

public ResourceConfigurer<HadoopConfigConfigurer> withResources()
                                                         throws java.lang.Exception

Description copied from interface: HadoopConfigConfigurer

Specify configuration options as resource properties with a ResourceConfigurer.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   Properties props = new Properties();
   config
     .withResources()
       .resource("cfg-1.properties")
       .resource("cfg-2.properties")
       .and();
 }
 

XML:

 <hadoop:configuration properties-location="cfg-1.properties, cfg-2.properties"/>
 

Specified by:

withResources in interface HadoopConfigConfigurer

Returns:

ResourceConfigurer for chaining

Throws:

java.lang.Exception - if error occurred

withProperties

public PropertiesConfigurer<HadoopConfigConfigurer> withProperties()
                                                            throws java.lang.Exception

Description copied from interface: HadoopConfigConfigurer

Specify configuration options as properties with a PropertiesConfigurer.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   Properties props = new Properties();
   config
     .withProperties()
       .properties(props)
       .property("myKey1", ",myValue1")
       .and();
 }
 

XML:

 <util:properties id="props" location="props.properties"/>
   <prop key="myKey1">myValue1</prop>
 </util:properties>
 <hadoop:configuration properties-ref="props"/>
 

Specified by:

withProperties in interface HadoopConfigConfigurer

Returns:

PropertiesConfigurer for chaining

Throws:

java.lang.Exception - if error occurred

withSecurity

public SecurityConfigurer<HadoopConfigConfigurer> withSecurity()
                                                        throws java.lang.Exception

Description copied from interface: HadoopConfigConfigurer

Specify security options with a SecurityConfigurer.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   config
     .withSecurity()
       .authMethod("kerberos")
       .namenodePrincipal("hdfs/myhost@LOCALDOMAIN")
       .rmManagerPrincipal("yarn/myhost@LOCALDOMAIN");
 }
 

XML:

No equivalent

Specified by:

withSecurity in interface HadoopConfigConfigurer

Returns:

SecurityConfigurer for chaining

Throws:

java.lang.Exception - if error occurred

fileSystemUri

public HadoopConfigConfigurer fileSystemUri(java.lang.String uri)

Description copied from interface: HadoopConfigConfigurer

Specify a Hdfs file system uri.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   config
     .fileSystemUri("hdfs://myhost:8020");
 }
 

XML:

 <hadoop:configuration file-system-uri="hdfs://myhost:8020"/>
 

Specified by:

fileSystemUri in interface HadoopConfigConfigurer

Parameters:

uri - The Hdfs uri

Returns:

HadoopConfigConfigurer for chaining

resourceManagerAddress

public HadoopConfigConfigurer resourceManagerAddress(java.lang.String address)

Description copied from interface: HadoopConfigConfigurer

Specify a Yarn resource manager address.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   config
     .resourceManagerAddress("myRmHost:8032");
 }
 

XML:

 <hadoop:configuration rm-manager-uri="myRmHost:8032"/>
 

Specified by:

resourceManagerAddress in interface HadoopConfigConfigurer

Parameters:

address - The Yarn resource manager address

Returns:

HadoopConfigConfigurer for chaining

jobHistoryAddress

public HadoopConfigConfigurer jobHistoryAddress(java.lang.String jobHistoryAddress)

Description copied from interface: HadoopConfigConfigurer

Specify a MapReduce job history address.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   config
     .jobHistoryAddress("myJobHistoryHost:10020");
 }
 

XML:

 <hadoop:configuration job-history-uri="myJobHistoryHost:10020"/>
 

Specified by:

jobHistoryAddress in interface HadoopConfigConfigurer

Parameters:

jobHistoryAddress - The Yarn resource manager address

Returns:

HadoopConfigConfigurer for chaining

loadDefaults

public HadoopConfigConfigurer loadDefaults(boolean loadDefaults)

Description copied from interface: HadoopConfigConfigurer

Specify if Hadoop Configuration is initially based on default values. Default is true.

JavaConfig:

 public void configure(HadoopConfigConfigurer config) throws Exception {
   config
     .loadDefaults(true);
 }
 

XML:
No equivalent

Specified by:

loadDefaults in interface HadoopConfigConfigurer

Parameters:

loadDefaults - The flag if defaults should be loaded

Returns:

HadoopConfigConfigurer for chaining

getProperties

public java.util.Properties getProperties()

Gets the Properties.

Returns:

the properties

getResources

public java.util.Set<org.springframework.core.io.Resource> getResources()

Gets the Resources.

Returns:

the resources

getSecurity

public HadoopSecurity getSecurity()

Gets the HadoopSecurity.

Returns:

the security