Building and Running CDAP Applications

In the examples, we refer to the CDAP Sandbox as CDAP, and the example code that is running on it as an application. We'll assume that you are running your application in the default namespace; if not, you will need to adjust commands accordingly. For example, in a URL such as:

http://localhost:11015/v3/namespaces/default/apps...

to use the namespace my_namespace, you would replace default with my_namespace:

http://localhost:11015/v3/namespaces/my_namespace/apps...

Accessing CLI, curl, and the CDAP Sandbox bin

  • For brevity in the commands given below, we will simply use cdap cli for the CDAP Command Line Interface. Substitute the actual path of ./<CDAP-HOME>/bin/cdap cli, or <CDAP-HOME>\bin\cdap.bat cli on Windows, as appropriate.

  • A Windows-version of the application curl is included in the CDAP Sandbox as libexec\bin\curl.exe; use it as a substitute for curl in examples.

  • If you add the CDAP Sandbox bin directory to your path, you can simplify the commands. From within the <CDAP-HOME> directory, enter:

$ cdap cli export PATH=${PATH}:`pwd`/bin

The Windows path has been augmented with a directory where the CDAP Sandbox includes Windows-versions of commands such as curl.

Note: There is an issue with running Microsoft Windows and using the CDAP Local Sandbox scripts when CDAP_HOME is defined as a path with spaces in it. Until this is addressed, do not use a path with space characters in it for CDAP_HOME.

Building an Example Application Artifact

From the example's project root (such as examples/<example-dir>), build an example with the Apache Maven command:

To build without running tests, use:

To build all the examples, switch to the main examples directory and run the Maven command:

Starting CDAP

Before running an example application, check that an instance of CDAP is running and available; if not, follow the instructions for Starting and Stopping CDAP Sandbox.

If you can reach the CDAP UI through a browser at http://localhost:11011/, CDAP is running.

Deploying an Application

Once CDAP is started, there are multiple ways to deploy an application using an example JAR:

The CLI can be accessed under Windows using the bin\cdap.bat cli script.

  • Using an application such as curl (a Windows-version is included in the CDAP Sandbox in libexec\bin\curl.exe):

Starting an Application's Programs

Once an application is deployed, there are multiple methods for starting an application's programs:

  • The CDAP UI.

  • The Command Line Interface to start a specific program of an application. (In each CDAP example, the CLI commands for that particular example are provided):

Parameter

Description

Parameter

Description

<program-type>

One of mapreduceservicesparkworker, or workflow.

<app-id>

Name of the application being called.

<program-id>

Name of the MapReduceservicesparkworker or workflow being called.

 

Parameter

Description

Parameter

Description

<app-id>

Name of the application being called.

<program-types>

An optional comma-separated list of program types (mapreduceservicesparkworker, or workflow) which will start all programs of those types; for example, specifying 'spark,workflow' will start all Spark programs and workflows in the application.

  • The Program Lifecycle of the Lifecycle Microservices to start the programs of an application using a program such as curl

Stopping an Application's Programs

Once an application is deployed, there are multiple ways for stopping an application's programs:

Parameter

Description

Parameter

Description

<program-type>

One of mapreduceservicesparkworker, or workflow.

<app-id>

Name of the application being called.

<program-id>

Name of the MapReduceservicesparkworker or workflow being called.

Parameter

Description

Parameter

Description

<app-id>

Name of the application being called.

<program-types>

An optional comma-separated list of program types (mapreduceservicesparkworker, or workflow) which will stop all programs of those types; for example, specifying 'spark,workflow' will stop all workflows in the application.

  • Use the Program Lifecycle of the Lifecycle Microservices to stop the programs of an application using a program such as curl.

Removing an Application

Once an application is "stopped", when all of its programs are stopped, you can remove the application using the CDAP UI.

Alternatively, you can also use the Command Line Interface:

Note that any datasets created or used by the application will remain, as they are independent of the application. Datasets can be deleted from the CDAP UI, or by using the Microservices, the Java Client API, or the Command Line Interface API.

The artifact used to create the application will also remain, as multiple applications can be created from the same artifact. Artifacts can be deleted using the UI, Microservices, the Java Client API, or the Command Line Interface API.

Created in 2020 by Google Inc.