The Spark Program in Scala action plugin is available in the Hub. |
Executes user-provided Spark code in Scala.
Use this plugin when you want arbitrary Spark code.
Property | Macro Enabled? | Description | |
---|---|---|---|
Main Class Name | Yes | Required. The fully qualified class name for the Spark application. It must either be an | |
Scala | Yes | Required. The self-contained Spark application written in Scala. For example, an application that reads from CDAP stream with name
| |
Dependencies | Yes | Optional. Extra dependencies for the Spark program. It is a ‘,’ separated list of URI for the location of dependency jars. A path can be ended with an asterisk ‘*’ as a wildcard, in which all files with extension ‘.jar’ under the parent path will be included. | |
Compile at Deployment Time | No | Optional. Specify whether the code will get validated during pipeline creation time. Setting this to Default is true. |
Refer to the CDAP documentation on the enhancements that CDAP brings to Spark.