With explore enabled on a secure cluster, I get an exception whenever the explore service tries to talk to the Hive metastore.
hive-site.xml has 3 settings added to enable auth
Then, when explore service starts up I see an exception when it tries to get databases.
Seems like the explore service is not kerberos authenticated.
ooook. When explore service starts up, it connects to the Hive metastore and tries to show the default database to check that everything is ok. This fails. It fails because the Hive metastore db connection to mysql has timed out. On my cluster mysql wait_timeout is set to 180 seconds. Hive uses bonecp for db connection pooling, which checks for idle connections every 4 hours (half of mysql's default wait_timeout of 8 hours).
The fix is to include a bonecp-config.xml in the Hive metastore's classpath with:
though I'm not sure why we didn't see this on other clusters. Could be that the behavior is slightly different on secure clusters. In any case, will add a note in the docs about this.
changing the pooling strategy back to dbcp also seems to work. In hive-site.xml, add
docs will be updated as part of