Service & MR Spec lack datasets

Description

Services and MapReduce programs' specification does not include the Datasets these program use when these programs utilize datasets by means other than the useDataset(s)(datasetName) APIs (i.e. via annotations).

This also makes it difficult to implement: https://issues.cask.co/browse/CDAP-1461

Release Notes

None

Activity

Show:
Sreevatsan Raman
February 22, 2015, 12:25 PM

Here is the output of Service spec and mapreduce spec. Is this still an issue?

curl -s http://localhost:10000/v2/apps/Wise/services/WiseService | python -mjson.tool
{
"className": "co.cask.cdap.apps.wise.WiseService",
"description": "",
"handlers": {
"PageViewCountHandler": {
"className": "co.cask.cdap.apps.wise.WiseService$PageViewCountHandler",
"datasets": [
"pageViewStore"
],
"description": "",
"endpoints": [
{
"method": "POST",
"path": "/ip/{ip}/count"
},
{
"method": "GET",
"path": "/ip/{ip}/count"
}
],
"name": "PageViewCountHandler",
"properties": {}
}
},
"instances": 1,
"name": "WiseService",
"resources": {
"memoryMB": 512,
"virtualCores": 1
},
"workers": {}
}

curl -s http://localhost:10000/v2/apps/Wise/mapreduce/BounceCountsMapReduce | python -m json.tool
{
"className": "co.cask.cdap.apps.wise.BounceCountsMapReduce",
"dataSets": [],
"description": "Bounce Counts MapReduce job",
"name": "BounceCountsMapReduce",
"outputDataSet": "bounceCountStore",
"properties": {}
}

Ali Anwar
February 22, 2015, 2:25 PM

Ok, I added a test case that fails for this reason:
https://github.com/caskdata/cdap/compare/mr-no-datasets

One possibility is that the AppWithMapreduce application is not a valid application (it uses annotation to declare dataset used, but doesn't use useDataset() API).

Bhooshan Mogal
February 24, 2015, 5:44 AM

So what's the status on this? Maybe once we're done with the major work for code freeze, we can check this with another App?

Ali Anwar
April 7, 2015, 7:51 AM

I verified with services and MR, and the specs returned show the datasets (similar to 's output above).
This is not a currently-existing issue.

Cannot Reproduce

Assignee

Ali Anwar

Reporter

Ali Anwar

Labels

Docs Impact

None

UX Impact

None

Components

Affects versions

Priority

Major