Goals

• The market will provide an application store for CDAP users to quickly and efficiently add capabilities into the platform.   
• The Cask Market will be a read only for 4.0 and include provisions for working offline.   

• As a CDAP User I want a plus button on the navbar so that I can an add new capabilities from anywhere in the platform
• As a CDAP User I want to be able to add new artifacts from the market into the system to speed up development and improve the overall developer experience.
• As a Hydrator User I want to be able to add plugins from the market to expand the capabilities of my pipeline.  
• As a Hydrator User from within the Hydrator Studio I want to be able to add new pipelines or applications from within Hydrator, and upon adding, remain in the Hydrator context.
• As a CDAP user I want to be able to add examples from the market.  
• As a CDAP user from within the market I want to be able to know what I've added into the platform and have those special cased when browsing the market.  
• AS a CDAP user I want to have access to the market in offline mode, so that I can add capabilities into the system from where-ever my cluster is deployed.  

Requirements

Market Requirements

• The market should be accessible from anywhere within CDAP.
• The market should support adding:
  • Artifacts
    • Applications
    • Native Plugins
    • Third Party Plugins
  • Pipelines
  • DataPacks
• The market should provide context when browsing to know if I've already added an entity.  I should still be able to create another instance and the market should keep track.  
• The market should instantiate the new entity wizard where applicable.  
• The market should be browsable independently of the product, from the website.  
• The market on the website should link to localhost and the cdap port to indicate "open in cdap" to add entities.  
• The market should be supported by default in both Distributed and Standalone, with the option to be disabled.  
• The market should work in offline mode after syncing the repository onto the cluster.  
• The market should be an independent entity outside of CDAP.
• CDAP should track what has been added previously
• The market should be read only for the first iteration.   

Market Repository Requirements

• The repo should include metadata only.
• The contract between the repo and the entities needs to be versioned to support changes in repo behavior or meta data specifications for entities.
• The repo should be updatable/installable from the UI, REST API or CMD Line.  
• The repo should include versioning information for entities.
• The repo should include a checksum for entities.
• The repo should include information for third party entities.
• The repo should include author and licensing information for entities.
• The repo should include dependency information for entities
• The repo should include abstraction information (service descriptor) for how an entity will interact with the market wizard workflow.   The wizard will consist of 1 to N steps and must be able to abstract all information necessary from the entity descriptor for how an entity is added including dependencies (internal/external) precedence of steps.

Market Datapack Requirements

Datapacks are sources of data that will improve the bootstrapping and getting started experience on the platform.  

• Datapacks should be able to be added for:
  • SQL datapacks 
    • MYSQL .sql files
    • SQLITE .sqlite
  • S3 datapacks
    • Text files
    • csv examples
  • HTTP (for hydrator realtime pipelines)
    • Endpoints for various supported web services
• Datapacks should be available to be added as new sources plugin templates in hydrator.  
• Datapacks should be able available for custom applications.