The JSVC_HOME in /etc/default/hadoop is incorrect on HDP 2.0 clusters (/usr/lib/bigtop-utils vs actual: /usr/libexec/bigtop-utils). This makes it impossible to spin up an HDP 2.0 cluster. We at least need to make it an attribute.
https://github.com/caskdata/hadoop_cookbook/pull/135/files fixed it for HDP 2.[1,2], probably broke it for 2.0
https://github.com/caskdata/hadoop_cookbook/pull/190/files hardcoded this change.
It's not an attribute. The package only installs it into one place, so no need to make it configurable.
You've verified that the directory is /usr/libexec/bigtop-utils on HDP 2.0 (which one?)... Let's check the rest. Also, this only affects secure clusters, which is probably how we missed this. Meh...
Also, can you verify where the jsvc binary comes from on that machine? What Linux distribution? I want to make sure that we're changing this correctly. I understood that it came from bigtop-jsvc in HDP-UTILS, which is the same no matter what version of HDP you use. The cookbook simply uses the same version of this repository for every HDP version.
on centos6.6, HDP 188.8.131.52, which came with bigtop-jsvc-1.0.10: