We're updating the issue view to help you get more done. 

Unable to run the recipe

Description

Installing Cookbook Gems:
Compiling Cookbooks...
[2017-08-22T17:59:15+05:30] FATAL: HDFS NameNode HA requires node['hadoop']['hdfs_site']['dfs.nameservices'] to be set

================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/hadoop/recipes/_hadoop_hdfs_ha_checkconfig.rb
================================================================================

SystemExit
---------- exit

Cookbook Trace:
--------------- /var/chef/cache/cookbooks/hadoop/recipes/_hadoop_hdfs_ha_checkconfig.rb:25:in `from_file'

Relevant File Content:
---------------------- /var/chef/cache/cookbooks/hadoop/recipes/_hadoop_hdfs_ha_checkconfig.rb:

18: #
19: # HDFS HA requires dfs.nameservices
20:
21:
22: if node['hadoop'].key?('hdfs_site') && node['hadoop']['hdfs_site'].key?('dfs.nameservices')
23: dfs_nameservices = node['hadoop']['hdfs_site']['dfs.nameservices'].split(',')
24: else
25>> Chef::Application.fatal!("HDFS NameNode HA requires node['hadoop']['hdfs_site']['dfs.nameservices'] to be set")
26: end
27:
28: # We have dfs.nameservices, and now need to check them
29: dfs_nameservices.each do |ns|
30: # Start namenode checks
31: next unless node['hadoop']['hdfs_site'].key?("dfs\.ha\.namenodes\.#{ns}")
32: # We need two and only two NameNodes
33: namenodes = node['hadoop']['hdfs_site']["dfs\.ha\.namenodes\.#{ns}"].split(',')
34: if namenodes.size != 2

System Info:
------------ chef_version=13.2.20
platform=ubuntu
platform_version=14.04
ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
program_name=chef-client worker: ppid=8621;start=17:59:14;
executable=/opt/chefdk/bin/chef-client

Running handlers:
[2017-08-22T17:59:15+05:30] ERROR: Running exception handlers
Running handlers complete
[2017-08-22T17:59:15+05:30] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 01 seconds
[2017-08-22T17:59:15+05:30] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2017-08-22T17:59:15+05:30] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-08-22T17:59:15+05:30] ERROR: exit
[2017-08-22T17:59:15+05:30] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Release Notes

None

Activity

Show:
Derek Wood
August 22, 2017, 9:58 PM

Hi,
This isn't an actual cookbook error, but rather a check to prevent it from rendering an invalid hdfs-site.xml configuration that you have provided. It seems you have specified

but have not specified the other necessary attributes for HA namenode. In particular

must be set.

I suggest taking a look at the _hadoop_hdfs_ha_checkconfig.rb recipe, or the Apache docs to see the other necessary configurations for HA Namenode.

saif ahmad
August 23, 2017, 4:46 AM

Hi,

Thanks for the reply.
Where do I set this value?

In which file? I know this question is very lame but I'm just a week old using chef and cookbooks.

Derek Wood
August 24, 2017, 5:32 PM

You would probably want to set it in a role, or another cookbook which "wraps" the hadoop cookbook. We have some notes on wrapping the cookbook here.

Since this recipe is being called, it means you are either 1) already setting node['hadoop']['hdfs_site']['dfs.ha.automatic-failover.enabled'] somewhere, or 2) directly running this recipe or hadoop_hdfs_zkfc via the runlist. If 1) you should set the other attributes in the same place. If 2), and also as a general recommendation, I would suggest not worrying about HA yet and first get it working with the simplest setup as you become more familiar with chef.

I'm going to go ahead and close this as it is not a bug.

Won't Fix

Assignee

Derek Wood

Reporter

saif ahmad

Labels

Docs Impact

None

UX Impact

None

Components

Priority

Critical
Configure