Apache Hadoop Key Management Server (KMS)

Owned by Robin Rielley
Last updated: Jun 07, 2022
1 min read

CDAP integrates with Apache Hadoop Key Management Server (KMS) as the backend for Secure Storage. To use this secure storage implementation, set security.store.provider to kms in cdap-site.xml.

Prerequisites

Since KMS is only available in Apache Hadoop as of version 2.6.0, this secure storage implementation can only be used on clusters with Apache Hadoop 2.6.0 or later installed.

The KMS path should be available as the hadoop.security.key.provider.path property of either core-site.xml or hdfs-site.xml on all cluster hosts. Refer to KMS Client Configuration for the expected format of the value of this property.

Additionally, the /etc/hadoop/kms-acls.xml file on the KMS host should be updated to include users with appropriate permissions.

  • If impersonation is enabled and KMS-backed secure storage is used from programs, the impersonated user should be provided appropriate permissions in the /etc/hadoop/kms-acls.xml.

  • If it is used through the Secure Storage Microservices, the CDAP logged-in user should be provided appropriate permissions in the /etc/hadoop/kms-acls.xml.

Created in 2020 by Google Inc.