org.springframework.yarn.am.allocate

Class DefaultContainerAllocator

java.lang.Object

org.springframework.yarn.support.LifecycleObjectSupport

org.springframework.yarn.am.allocate.AbstractAllocator

org.springframework.yarn.am.allocate.AbstractPollingAllocator

org.springframework.yarn.am.allocate.DefaultContainerAllocator

All Implemented Interfaces:

org.springframework.beans.factory.Aware, org.springframework.beans.factory.BeanFactoryAware, org.springframework.beans.factory.InitializingBean, org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle, ContainerAllocator

public class DefaultContainerAllocator
extends AbstractPollingAllocator
implements ContainerAllocator

Default allocator which polls resource manager, requests new containers and acts as a heart beat sender at the same time.

Author:

Janne Valkealahti

Constructor Summary

Constructors

Constructor and Description
DefaultContainerAllocator()

Method Summary

Methods

Modifier and Type	Method and Description
void	addListener(ContainerAllocatorListener listener) Adds the ContainerAllocatorListener.
void	allocateContainers(ContainerAllocateData containerAllocateData) Allocate new containers.
void	allocateContainers(int count) Allocate new containers.
protected org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse	doContainerRequest() Subclasses needs to implements this method to do container requests against resource manager.
int	getMemory() Gets the memory for container request.
int	getPriority() Gets the priority for container request.
int	getVirtualcores() Gets the virtualcores for container request.
protected void	handleAllocatedContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers) Subclasses needs to implement this method to handle newly allocated containers.
protected void	handleCompletedContainers(java.util.List<org.apache.hadoop.yarn.api.records.ContainerStatus> containerStatuses) Subclasses needs to implement this method to handle newly released containers.
boolean	isLocality() Checks if is locality relax flag is enabled.
protected void	onInit() Subclasses may implement this for initialization logic.
protected java.util.List<org.apache.hadoop.yarn.api.records.Container>	preProcessAllocatedContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers) Pre-process allocated containers.
void	releaseContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Release container.
void	releaseContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers) Release containers.
void	setAllocationValues(java.lang.String id, java.lang.Integer priority, java.lang.Integer virtualcores, java.lang.Integer memory, java.lang.Boolean locality) Sets the allocation values for given identifier.
void	setLocality(boolean locality) Sets the flag telling if resource allocation should not be relaxed.
void	setMemory(int memory) Sets the memory for container request defining memory of the resource.
void	setPriority(int priority) Sets the priority for container request.
void	setProgress(float progress) Sets the current progress of application.
void	setVirtualcores(int virtualcores) Sets the virtualcores for container request defining number of virtual cpu cores of the resource.

Methods inherited from class org.springframework.yarn.am.allocate.AbstractPollingAllocator

doStart, doStop, populateNmTokenCache, setTrigger

Methods inherited from class org.springframework.yarn.am.allocate.AbstractAllocator

getApplicationAttemptId, getConfiguration, getEnvironment, getRmTemplate, setApplicationAttemptId, setConfiguration, setEnvironment, setRmTemplate

Methods inherited from class org.springframework.yarn.support.LifecycleObjectSupport

afterPropertiesSet, getBeanFactory, getPhase, getTaskExecutor, getTaskScheduler, getYarnEventPublisher, isAutoStartup, isRunning, setAutoStartup, setBeanFactory, setPhase, setTaskExecutor, setTaskScheduler, setYarnEventPublisher, start, stop, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultContainerAllocator

public DefaultContainerAllocator()

Method Detail

onInit

protected void onInit()
               throws java.lang.Exception

Description copied from class: LifecycleObjectSupport

Subclasses may implement this for initialization logic. Called during the InitializingBean phase. Implementor should always call super method not to break initialization chain.

Overrides:

onInit in class AbstractPollingAllocator

Throws:

java.lang.Exception - exception

allocateContainers

public void allocateContainers(int count)

Description copied from interface: ContainerAllocator

Allocate new containers.

Specified by:

allocateContainers in interface ContainerAllocator

Parameters:

count - the new container count to allocate

addListener

public void addListener(ContainerAllocatorListener listener)

Description copied from interface: ContainerAllocator

Adds the ContainerAllocatorListener.

Specified by:

addListener in interface ContainerAllocator

Parameters:

listener - the ContainerAllocatorListener

allocateContainers

public void allocateContainers(ContainerAllocateData containerAllocateData)

Description copied from interface: ContainerAllocator

Allocate new containers.

Specified by:

allocateContainers in interface ContainerAllocator

Parameters:

containerAllocateData - the ContainerAllocateData

releaseContainers

public void releaseContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers)

Description copied from interface: ContainerAllocator

Release containers.

Specified by:

releaseContainers in interface ContainerAllocator

Parameters:

containers - the containers

releaseContainer

public void releaseContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: ContainerAllocator

Release container.

Specified by:

releaseContainer in interface ContainerAllocator

Parameters:

containerId - the container id

doContainerRequest

protected org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse doContainerRequest()

Description copied from class: AbstractPollingAllocator

Subclasses needs to implements this method to do container requests against resource manager. This method is called during the polling cycle handled by this class. New containers and newly released containers are passed to methods AbstractPollingAllocator.handleAllocatedContainers(List) and AbstractPollingAllocator.handleCompletedContainers(List).

Specified by:

doContainerRequest in class AbstractPollingAllocator

Returns:

AllocateResponse from a resource manager

preProcessAllocatedContainers

protected java.util.List<org.apache.hadoop.yarn.api.records.Container> preProcessAllocatedContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers)

Description copied from class: AbstractPollingAllocator

Pre-process allocated containers. Allows implementors to intercept containers before further processing is done. Default implementation returns list as it is.

Overrides:

preProcessAllocatedContainers in class AbstractPollingAllocator

Parameters:

containers - the containers

Returns:

the list of containers

handleAllocatedContainers

protected void handleAllocatedContainers(java.util.List<org.apache.hadoop.yarn.api.records.Container> containers)

Description copied from class: AbstractPollingAllocator

Subclasses needs to implement this method to handle newly allocated containers.

Specified by:

handleAllocatedContainers in class AbstractPollingAllocator

Parameters:

containers - list of newly allocate containers

handleCompletedContainers

protected void handleCompletedContainers(java.util.List<org.apache.hadoop.yarn.api.records.ContainerStatus> containerStatuses)

Description copied from class: AbstractPollingAllocator

Subclasses needs to implement this method to handle newly released containers.

Specified by:

handleCompletedContainers in class AbstractPollingAllocator

Parameters:

containerStatuses - list of newly released containers

setProgress

public void setProgress(float progress)

Description copied from interface: ContainerAllocator

Sets the current progress of application.

Specified by:

setProgress in interface ContainerAllocator

Parameters:

progress - the current progress of application

setAllocationValues

public void setAllocationValues(java.lang.String id,
                       java.lang.Integer priority,
                       java.lang.Integer virtualcores,
                       java.lang.Integer memory,
                       java.lang.Boolean locality)

Sets the allocation values for given identifier.

Parameters:

id - the allocation identifier

priority - the base priority

virtualcores - the cpu count

memory - the memory

locality - the locality flag

getPriority

public int getPriority()

Gets the priority for container request.

Returns:

the priority

setPriority

public void setPriority(int priority)

Sets the priority for container request.

Parameters:

priority - the new priority

getVirtualcores

public int getVirtualcores()

Gets the virtualcores for container request.

Returns:

the virtualcores

setVirtualcores

public void setVirtualcores(int virtualcores)

Sets the virtualcores for container request defining number of virtual cpu cores of the resource.

Parameters:

virtualcores - the new virtualcores

getMemory

public int getMemory()

Gets the memory for container request.

Returns:

the memory

setMemory

public void setMemory(int memory)

Sets the memory for container request defining memory of the resource.

Parameters:

memory - the new memory

isLocality

public boolean isLocality()

Checks if is locality relax flag is enabled.

Returns:

true, if is locality is enabled.

setLocality

public void setLocality(boolean locality)

Sets the flag telling if resource allocation should not be relaxed. Setting this to true means that relaxing is not used. Default value is false.

Parameters:

locality - the new locality relax flag