This task is intended to launch a Spark application. The task submits the Spark application for local execution. This task is appropriate for a local deployment where any local file references can be resolved. It is not meant for any type of cluster deployments.
The spark-client task has the following options:
- spark.app-args
- The arguments for the Spark application. (String[], default:
[]) - spark.app-class
- The main class for the Spark application. (String, default:
<none>) - spark.app-jar
- The path to a bundled jar that includes your application and its dependencies, excluding any Spark dependencies. (String, default:
<none>) - spark.app-name
- The name to use for the Spark application submission. (String, default:
<none>) - spark.executor-memory
- The memory setting to be used for each executor. (String, default:
1024M) - spark.master
- The master setting to be used (local, local[N] or local[*]). (String, default:
local) - spark.resource-archives
- A comma separated list of archive files to be included with the app submission. (String, default:
<none>) - spark.resource-files
- A comma separated list of files to be included with the application submission. (String, default:
<none>)