Overview

The purpose of this page is to illustrate the plan for ApplicationTemplate and Application consolidation.  This work is being tracked in 

Why do we want to consolidate templates and applications? In CDAP 3.0, an ApplicationTemplate is a way for somebody to write an Application that can be given some configuration to create an Adapter. The story is confusing; one would expect an ApplicationTemplate to create... Applications. Instead, we use the term Adapter because Application means something else already. In addition an ApplicationTemplate can only include a single workflow or a single worker, giving people different experiences for templates and applications.

Really, the goal of templates was to be able to write one piece of Application code that could be used to create multiple Applications. To do this requires that an Application can be configured at creation time instead of at compile time. For example, a user should be able to set the name of their dataset based on configuration instead of hardcoding it in the code. To support this, we plan on making it possible to get a configuration object from the ApplicationContext available in Application's configure() method. This allows somebody to pass in a config when creating an Application through the RESTful API, which can be used to configure an Application. The relevant programmatic API changes are shown below, with an example of how they might be used:

public interface ApplicationContext<T extends Config> {
  T getConfig();
}

public interface Application<T extends Config> {
  void configure(ApplicationConfigurer configurer, ApplicationContext<T> context);
}
 
public abstract class AbstractApplication<T> implements Application<T extends Config> {
  ...
  protected final ApplicationContext<T> getContext() { return context; }
}
 
public class MyApp extends AbstractApplication<MyApp.MyConfig> {
 
  public static class MyConfig extends Config {
    @Nullable
    private String stream;
 
    @Nullable
    private String table;
    private MyFlowConfig flowOptions;
 
    private MyConfig() {
      this.stream = "A";
      this.table = "X";
    }
  }
 
  public void configure() {
    // ApplicationContext now has a method to get a custom config object whose fields will
    // be injected using the values given in the RESTful API
    MyConfig config = getContext().getConfig();
    addStream(new Stream(config.stream));
    createDataset(config.table, Table.class);
    addFlow(new MyFlow(config.stream, config.table, config.flowOptions));
  }
}

Story Details

Deploying an Application

Users will still be able to deploy an app in one call. Suppose a user wants to deploy their application contained in myapp-1.0.0.jar.  They make the same RESTful call they would today:

PUT /namespaces/default/apps/myapp -H "X-Archive-Name: myapp-1.0.0.jar" -H "Content-Type: application/octet-stream" --data-binary @myapp-1.0.0.jar

Internally, CDAP will add the jar to its ArtifactRepository (new in CDAP 3.1), and then create an application from that artifact. In this example, the application creates stream A and Table X by default.

After the app is deployed, a user can now create Application myapp2 by referencing the artifact and config in their request without actually including the jar contents in the request. This lets users create applications using only config.

PUT /namespaces/default/apps/myapp2 -H "Content-Type: application/json" -d '{ "artifact": { "name": "myapp", "version": "1.0.0" }, "config": { "stream": "B", "table": "X" } }'

Deploying an Artifact, then an Application

Users can also deploy an artifact without creating an application.

POST /namespaces/default/artifacts/myapp --data-binary @myapp-1.0.1.jar

An application can then be created from that artifact in a separate call.

PUT /namespaces/default/apps/myapp3 -H "Content-Type: application/json" -d '{ "artifact": { "name": "my-app", "version": "1.0.1" }, "config": {"stream": "C", "table": "X"} }'

Updating an Application

Users will also be able to update their applications to use a different version of an artifact.

PUT /namespaces/default/apps/myapp/properties -H "Content-Type: application/json" -d '{ "artifact": { "name":"myapp", "version":"1.0.1" }, "config": { "stream": "A", "table": "X" } }'

System Artifacts

System artifacts are special artifacts that can be accessed in other namespaces. They cannot be deployed through the RESTful API. Instead, they are placed in a directory on the CDAP master host. When CDAP starts up, the directory will be scanned and those artifacts will be added to the system. Example uses for system artifacts are the ETLBatch and ETLRealtime applications that we want to include out of the box. When a user wants to create an application from a system artifact, they make the same RESTful call as before, except adding the namespace to the artifact section of the call:

PUT /namespaces/default/apps/somePipeline -H "Content-Type: application/json" -d '{ "artifact": { "namespace": "system", "name":"ETLBatch", "version":"3.1.0" }, "config": { ... } }'

RESTful API changes

Application APIs

Artifact APIs

{
  "className": "co.cask.cdap.example.MyPlugin",
  "description": "My Plugin",
  "name": "MyPlugin",
  "properties": {
    "threshold": { "name": "thresh", "type": "int", "required": false },
    "user": { "name": "user", "type": "config", "required": true,
      "fields": {
        "id": { "name": "id", "type": "long", "required": true },
        "digits": { "name": "phoneNumber", "type": "string", "required": true }
      }
    }
  }
}

Template APIs (will be removed)