Creating Compute Profiles

Compute profiles tell CDAP which provisioner to use when creating a cluster and specify the cluster configuration. They also specify the provisioner configuration that should be used when creating a cluster.

Native Compute Profile

The native profile is a special profile that runs programs natively in the same environment that CDAP runs in. Before Cloud Runtimes were introduced, this was the only way to run programs. With the CDAP Sandbox, this means programs run in the same JVM as CDAP. With CDAP Distributed, this means programs run in the same YARN cluster that CDAP is running in.

Scope

Each compute profile has a scope: system or user. You can use system compute profiles in any namespace. User compute profiles exist within a namespace, and only programs in that namespace can use user compute profiles.

To create a system compute profile, navigate to the CDAP administration page. This page includes a list of all system compute profiles and the button to create a system compute profile.

To create a user compute profile, navigate to the CDAP administration page and then navigate to the namespace that you want to create the profile in. From there, you can create a profile that exists only within that namespace.

Creation

To create a compute profile, first select the provisioner that the profile uses to create and configure the cloud runtime.

After you select a provisioner, you need to provide the configuration that the provisioner requires.

Note that the fields marked with the shields contain sensitive information, such as secret keys. We recommend that you provide sensitive information through the CDAP Secure Storage API, and you can do this by adding a secure key with the Security Microservices. Then, click the shield icon in the UI to select a secure key or, in the case of system compute profiles, type the name of the key. CDAP adds the key as a macro in the form of ${secure(<secure-key-name>)}.

In general, provisioner configuration can be overridden at runtime. To lock certain settings so that they cannot be overridden, click the lock symbol to make the value immutable.