Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

Overview

The purpose of this page is to illustrate the plan for ApplicationTemplate and Application consolidation.  This work is being tracked in 

Error rendering macro 'jira' : Unable to locate Jira server for this macro. It may be due to Application Link configuration.
.

 

Why do we want to consolidate templates and applications? In CDAP 3.0, an ApplicationTemplate is a way for somebody to write an Application that can be given some configuration to create an Adapter. The story is confusing; one would expect an ApplicationTemplate to create... Applications. Instead, we use the term Adapter because Application means something else already. In addition an ApplicationTemplate can only include a single workflow or a single worker, giving people different experiences for templates and applications.

 

Really, the goal of templates was to be able to write one piece of Application code that could be used to create multiple Applications. To do this requires that an Application can be configured at creation time instead of at compile time. For example, a user should be able to set the name of their dataset based on configuration instead of hardcoding it in the code. To support this, we plan on making it possible to get a configuration object from the ApplicationContext available in Application's configure() method. This allows somebody to pass in a config when creating an Application through the RESTful API, which can be used to configure an Application. The relevant programmatic API changes are shown below, with an example of how they might be used:

 

public interface ApplicationContext<T extends Config> {
  T getConfig();
}

public interface Application<T extends Config> {
  void configure(ApplicationConfigurer configurer, ApplicationContext<T> context);
}
 
public abstract class AbstractApplication<T> implements Application<T extends Config> {
  ...
  protected final ApplicationContext<T> getContext() { return context; }
}
 
public class MyApp extends AbstractApplication<MyApp.MyConfig> {
 
  public static class MyConfig extends Config {
    @Nullable
    private String stream;
 
    @Nullable
    private String table;
    private MyFlowConfig flowOptions;
 
    private MyConfig() {
      this.stream = "A";
      this.table = "X";
    }
  }
 
  public void configure() {
    // ApplicationContext now has a method to get a custom config object whose fields will
    // be injected using the values given in the RESTful API
    MyConfig config = getContext().getConfig();
    addStream(new Stream(config.stream));
    createDataset(config.table, Table.class);
    addFlow(new MyFlow(config.stream, config.table, config.flowOptions));
  }
}

 

Story Details

 

Users will still be able to deploy an app in one call. Suppose a user wants to deploy their application contained in myapp-1.0.0.jar.  They make the same RESTful call they would today:

PUT /namespaces/default/apps/myapp -H "X-Archive-Name: myapp-1.0.0.jar" -H "Content-Type: application/octet-stream" --data-binary @myapp-1.0.0.jar

 

Internally, CDAP will add the jar to its ArtifactRepository (new in CDAP 3.1), and then create an application from that artifact. In this example, the application creates stream A and Table X by default.

After the app is deployed, a user can now create Application myapp2 by referencing the artifact and config in their request without actually including the jar contents in the request. This lets users create applications using only config.

PUT /namespaces/default/apps/myapp2 -H "X-Archive-Name: myapp-1.0.0.jar" -H "Content-Type: application/json" -d '{ "stream": "B", "table": "X" }'

 

Users can also deploy an artifact without creating an application.

PUT /namespaces/default/artifacts/myapp/versions/1.0.1 --data-binary @myapp-1.0.1.jar

 

An application can then be created from that artifact in a separate call.

PUT /namespaces/default/apps/myapp3 -H "Content-Type: application/json" -d '{ "artifact": { "name": "my-app", "version": "1.0.1" }, "config": {"stream": "C", "table": "X"} }'

 

 

Users will also be able to update their applications to use a different version of an artifact.

PUT /namespaces/default/apps/myapp/properties -H "Content-Type: application/json" -d '{ "artifact": { "name":"myapp", "version":"1.0.1" }, "config": { "stream": "A", "table": "X" } }'

 

RESTful API changes

Application APIs

TypePathBodyHeadersDescription
GET/v3/namespaces/<namespace-id>/apps?label=<label>  for example, to get all "ETLBatch" applications
PUT/v3/namespaces/<namespace-id>/apps/<app-name>{ 'artifact': 'name-version', 'config': { ... } }Content-Type: application/json

create an application from an existing artifact.

Note: Edits existing API, different behavior based on content-type

PUT/v3/namespaces/<namespace-id>/apps/<app-name>/properties{ 'artifact': 'name-version', 'config': { ... } } update an existing application. No programs can be running

Artifact APIs

TypePathBodyDescription
GET/v3/namespaces/<namespace-id>/artifacts  
GET/v3/namespaces/<namespace-id>/artifacts/<artifact-name> Get data about all artifact versions
GET/v3/namespaces/<namespace-id>/artifacts/<artifact-name>/versions/<version> Get details about the artifact, such as what plugins and applications are in the artifact and properties they support
PUT/v3/namespaces/<namespace-id>/artifacts/<artifact-name>/versions/<version>jar contentsAdd a new artifact
GET/v3/namespaces/<namespace-id>/extensions 

 

GET/v3/namespaces/<namespace-id>/extensions/<plugin-type>  
GET/v3/namespaces/<namespace-id>/extensions/<plugin-type>/plugins/<plugin-name> 

config properties can be nested now. For example:

{
  "className": "co.cask.cdap.example.MyPlugin",
  "description": "My Plugin",
  "name": "MyPlugin",
  "properties": {
    "threshold": { "name": "thresh", "type": "int", "required": false },
    "user": { "name": "user", "type": "config", "required": true,
      "fields": {
        "id": { "name": "id", "type": "long", "required": true },
        "digits": { "name": "phoneNumber", "type": "string", "required": true }
      }
    }
  }
}

Template APIs (will be removed)

TypePathReplaced By
GET/v3/templates 
GET/v3/templates/<template-name> 
GET/v3/templates/<template-name>/extensions/<plugin-type>/v3/namespaces/<namespace-id>/extensions/<plugin-type>
GET/v3/templates/<template-name>/extensions/<plugin-type>/plugins/<plugin-name>/v3/namespaces/<namespace-id>/extensions/<plugin-type>/plugins/<plugin-name>
PUT/v3/namespaces/<namespace-id>/templates/<template-id> 
GET/v3/namespaces/<namespace-id>/adapters 
GET/v3/namespaces/<namespace-id>/adapters/<adapter-name> 
POST/v3/namespaces/<namespace-id>/adapters/<adapter-name>/start 
POST/v3/namespaces/<namespace-id>/adapters/<adapter-name>/stop 
GET/v3/namespaces/<namespace-id>/adapters/<adapter-name>/status 
GET/v3/namespaces/<namespace-id>/adapters/<adapter-name>/runs 
GET/v3/namespaces/<namespace-id>/adapters/<adapter-name>/runs/<run-id> 
DELETE/v3/namespaces/<namespace-id>/adapters/<adapter-name> 
  • No labels