Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Really, the goal of templates was to be able to write one piece of Application code that could be used to create multiple Applications. To do this requires that an Application can be configured at creation time instead of at compile time. For example, a user should be able to set the name of their dataset based on configuration instead of hardcoding it in the code. To support this, we plan on making it possible to get a configuration object from the ApplicationContext available in Application's configure() method. This allows somebody to pass in a config when creating an Application through the RESTful API, which can be used to configure an Application. The relevant programmatic API changes are shown below, with an example of how they might be used:

 

Code Block
//-------------- CDAP API changes --------------
public interface ApplicationContext<T extends Config> {
  T getConfig();
}

public interface Application<T extends Config> {
  void configure(ApplicationConfigurer configurer, ApplicationContext<T> context);
}
 
public abstract class AbstractApplication<T> implements Application<T extends Config> {
  ...
  protected final ApplicationContext<T> getContext() { return context; }
}
 
public class //-------------- Example Application --------------
public class MyApp extends AbstractApplication<MyApp.MyConfig> {
 
  public static class MyConfig extends Config {
    @Nullable
    @Description("The name privateof Stringthe stream;  to read from. Defaults  @Nullableto 'A'.")
    private String tablestream;
 
   private MyFlowConfig@Nullable
flowOptions;       private MyConfig() {
      this.stream = "A";
      this.table = "X";@Description("The name of the table to write to. Defaults to 'X'.")
    private String table;
 
    @Name("flow")
    }private MyFlowConfig flowConfig;
} 
    public voidprivate configureMyConfig() {
    // ApplicationContext nowthis.stream has a method to get a custom config object whose fields will= "A";
      this.table = "X";
    }
// be injected}
using 
the values givenpublic invoid theconfigure() RESTful{
API    // MyConfig config = getContext().getConfig();
    addStream(new Stream(config.stream));
    createDataset(config.table, Table.class);
    addFlow(new MyFlow(config.stream, config.table, config.flowOptions));ApplicationContext now has a method to get a custom config object whose fields will
    // be injected using the values given in the RESTful API
    MyConfig config = getContext().getConfig();
    addStream(new Stream(config.stream));
    createDataset(config.table, Table.class);
    addFlow(new MyFlow(config.stream, config.table, config.flowConfig));
  }
}
 
public class MyFlow implements Flow {
  @Property
  private String stream;
  @Property
  private String table;
  @Property
  private FlowConfig flowConfig;
 
  public static final FlowConfig extends Config {
    private ReaderConfig reader;
    private WriterConfig writer;
  }
 
  MyFlow(String stream, String table, FlowConfig flowConfig) {
    this.stream = stream;
    this.table = table;
    this.flowConfig = flowConfig;
  }
 
  @Override
  public FlowSpecification configure() {
    return FlowSpecification.Builder.with()
      .setName("MyFlow")
      .setDescription("Reads from a stream and writes to a table")
      .withFlowlets()
        .add("reader", new StreamReader(flowConfig.reader))
        .add("writer", new TableWriter(flowConfig.writer))
      .connect()
        .fromStream(stream).to("reader")
        .from("reader").to("writer")
      .build();
  }
} 
 
public class StreamReader extends AbstractFlowlet {
  private OutputEmitter<Put> emitter;
 
  @ProcessInput
  public void process(StreamEvent event) {
  }
}

 

Use Case Walkthrough

1. Deploying an Artifact

User builds their application jar the same way they build it today. They make a call to deploy their artifact (jar).

Code Block
POST /namespaces/default/artifacts/myapp --data-binary @myapp-1.0.0.jar

CDAP opens the jar, figures out the bundle-version as the artifact version, figures out what apps, programs, datasets, and plugins are in the artifact, then stores the artifact on the filesystem and metadata in a table.

The user can examine the metadata by making a call:

Code Block
GET /namespaces/default/artifacts/myapp/versions/1.0.0
 
{
  "name": "purchase",
  "version": "3.1.0",
  "meta": {
    "created": "1234567890000",
    ...
  },
  "classes": {
    "apps": [
      {
        "className": "co.cask.cdap.examples.myapp.MyApp",
        "properties": {
          "stream": { 
            "name": "stream", 
            "description": "The name of the stream to read from. Defaults to 'A'.", 
            "type": "string", 
            "required": false 
          },
          "table": {
            "name": "table",
            "description": "The name of the table to write to. Defaults to 'X'.",
            "type": "string",
            "required": false,
          },
          "flowConfig": {
            "name": "flow",
            "description": "",
            "type": "config",
            ""            
          }
            "fields": {
            "id": { "name": "id", "type": "long", "required": true },
            "digits": { "name": "phoneNumber", "type": "string", "required": true }
          }
        }
      }
    ],
    "plugins": [
    ]
  }
}

 

...

Deploying an Application

Users will still be able to deploy an app in one call. Suppose a user wants to deploy their application contained in myapp-1.0.0.jar.  They make the same RESTful call they would today:

...