CDAP Microservices Guide

Owned by Robin Rielley
Last updated: Feb 11, 2021
1 min read

The Cask Data Application Platform (CDAP) enables Microservices through an HTTP interface for multitude of purposes: from deploying custom applications through checking the status of system and custom CDAP services. V3 of the Microservices includes the namespacing of applications, data, and metadata to achieve application and data isolation. This is an initial step towards introducing multi-tenancy into CDAP.

Introduction

  • Introduction: Conventions, converting from CDAP Microservices v2, naming restrictions, status codes, and working with CDAP security

General APIs

  • Namespace: Creating and managing namespaces

  • Metadata: Setting, retrieving, and deleting user metadata annotations

  • Preferences: Setting, retrieving, and deleting preferences

  • Configuration: Retrieving the CDAP and HBase configurations

  • Security: Granting, revoking and listing privileges on CDAP entities, managing secure storage

  • Transactions: Interacting with the transaction service

Major CDAP Entities APIs

  • Artifact: Deploying artifacts and retrieving details about plugins available to artifacts and classes contained within artifacts

  • Lifecycle: Deploying and managing applications, and managing the lifecycle of MapReduce programs, Spark programs, workflows, and custom services

  • Pipeline: Creating, updating, retrieving details of data pipelines in Draft mode.

  • Profile: Management of cloud runtime profiles

  • Dataset: Interacting with datasets, dataset modules, and dataset types

  • Service: Supports making requests to the methods of an application’s services

  • Workflow: Retrieving values from workflow tokens and statistics on workflow runs

Logging, Metrics, and Monitoring APIs

  • Logging: Retrieving application logs

  • Metrics: Metrieving metrics for system and user applications (user-defined metrics)

  • Monitor: Checking the status of various system and custom CDAP services

  • Dashboard: Check in real time status of past program runs and predict future resource usage

  • Reports: Generate reports to understand and monitor program runs and their performance

Created in 2020 by Google Inc.