By default, every application in Cloud Foundry starts with 1G disk quota and this can be adjusted to a default maximum of 2G. The default maximum can also be overridden up to 10G via Pivotal Cloud Foundry’s (PCF) Ops Manager GUI.
This configuration is relevant for Spring Cloud Data Flow because every stream and task deployment is composed of applications (typically Spring Boot uber-jar’s) and those applications are resolved from a remote maven repository. After resolution, the application artifacts are downloaded to the local Maven Repository for caching/reuse. With this happening in the background, there is a possibility the default disk quota (1G) fills up rapidly; especially, when we are experimenting with streams that are made up of unique applications. In order to overcome this disk limitation and depending on your scaling requirements,you may want to change the default maximum from 2G to 10G. Let’s review the steps to change the default maximum disk quota allocation.
From PCF’s Ops Manager, Select "Pivotal Elastic Runtime" tile and navigate to "Application Developer Controls" tab. Change the "Maximum Disk Quota per App (MB)" setting from 2048 to 10240 (10G). Save the disk quota update and hit "Apply Changes" to complete the configuration override.
Once the disk quota change is applied successfully and assuming you’ve a running application,
you may scale the application with a new disk_limit through CF CLI.
→ cf scale dataflow-server -k 10GB Scaling app dataflow-server in org ORG / space SPACE as user... OK .... .... .... .... state since cpu memory disk details #0 running 2016-10-31 03:07:23 PM 1.8% 497.9M of 1.1G 193.9M of 10G
→ cf apps Getting apps in org ORG / space SPACE as user... OK name requested state instances memory disk urls dataflow-server started 1/1 1.1G 10G dataflow-server.apps.io
Even when configuring the Data Flow server to use 10G of space, there is the possibility of exhausting the available space on the local disk. The server implements a least recently used (LRU) algorithm that will remove maven artifacts from the local maven repository. This is configured using the following configuration property, the default value is 25.
# The low water mark percentage, expressed as in integer between 0 and 100, that triggers cleanup of # the local maven repository # (for setting env var use SPRING_CLOUD_DATAFLOW_SERVER_CLOUDFOUNDRY_FREE_DISK_SPACE_PERCENTAGE) spring.cloud.dataflow.server.cloudfoundry.freeDiskSpacePercentage=25