Setting up Apache Sentry from source on a Coopr CDH cluster

Background

Since no released version of CDH supports the version of Apache Sentry that CDAP is integrating with, we cannot use Cloudera Manager to set up Sentry for testing. This page lists the steps to build Sentry from source and start the on a CDH cluster created using Coopr.

Prereqs

  1. git
  2. mvn

Setting up MySQL (Optional)

Create User

mysql> CREATE USER 'sentry'@'localhost' IDENTIFIED BY 'sentry';
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE DATABASE sentry_metastore;
Query OK, 1 row affected (0.00 sec)
mysql> GRANT ALL ON sentry_metastore.* TO 'sentry'@'localhost';
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye

Verify

# mysql -usentry -hlocalhost -psentry
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 17
Server version: 5.1.73-log Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use sentry_metastore;
Database changed

Steps

      1. Clone Apache Sentry source from https://github.com/apache/incubator-sentry
      2. There is a problem compiling the sentry-tests module right now, so comment it out temporarily from the root pom.xml until we find a fix
      3. mvn clean install -DskipTests will generate a sentry distribution under sentry-dist/target/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/. Only takes a few mins.
      4. cd sentry-dist/target/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/
      5. Create a sentry-site.xml in the conf directory with the following contents. This is a bare-minimum sentry-site.xml without kerberos. 

        <?xml version="1.0"?>
        <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
        <configuration>
          <property>
            <name>sentry.service.security.mode</name>
            <value>none</value>
          </property>
          <property>
            <name>sentry.service.admin.group</name>
            <value>admin1</value>
          </property>
          <property>
            <name>sentry.service.allow.connect</name>
            <value>impala,hive</value>
          </property>
          <property>
            <name>sentry.store.jdbc.url</name>
            <value>jdbc:derby:;databaseName=metastore_db;create=true</value>
          </property>
          <property>
            <name>sentry.store.jdbc.driver</name>
            <value>org.apache.derby.jdbc.EmbeddedDriver</value>
          </property>
          <property>
            <name>sentry.store.jdbc.password</name>
            <value>sentry</value>
          </property>
        </configuration>
      6. Set up the sentry policy store using a Derby DB

        bin/sentry --command schema-tool --conffile conf/sentry-site.xml --dbType derby --initSchema

        Or, for MySQL:

        bin/sentry --command schema-tool --conffile conf/sentry-site.xml --dbType mysql --initSchema --verbose

        Note: mysql jdbc driver jar should be in sentry/lib. Also, sentry does not accept a user without password for mysql (and perhaps postgresql etc)

      7. Start the sentry service 

        bin/sentry --command service --conffile conf/sentry-site.xml &> sentry-service.log &
      8. If you get the following error, it is because of an old metrics-core.jar in the hadoop classpath. If you see this, remove the file /usr/lib/hadoop-mapreduce/metrics-core-3.0.2.jar until we fix the error, and restart sentry using the above step. 

        Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: com.codahale.metrics.JmxAttributeGauge.<init>(Ljavax/management/MBeanServerConnection;Ljavax/management/ObjectName;Ljava/lang/String;)V
        	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        	at java.util.concurrent.FutureTask.get(FutureTask.java:188)
        	at org.apache.sentry.service.thrift.SentryService.waitOnFuture(SentryService.java:311)
        	at org.apache.sentry.service.thrift.SentryService$CommandImpl.run(SentryService.java:397)
        	at org.apache.sentry.SentryMain.main(SentryMain.java:114)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
        Caused by: java.lang.NoSuchMethodError: com.codahale.metrics.JmxAttributeGauge.<init>(Ljavax/management/MBeanServerConnection;Ljavax/management/ObjectName;Ljava/lang/String;)V
        	at com.codahale.metrics.jvm.BufferPoolMetricSet.getMetrics(BufferPoolMetricSet.java:45)
        	at org.apache.sentry.provider.db.service.thrift.SentryMetrics.registerMetricSet(SentryMetrics.java:149)
        	at org.apache.sentry.provider.db.service.thrift.SentryMetrics.<init>(SentryMetrics.java:97)
        	at org.apache.sentry.provider.db.service.thrift.SentryMetrics.getInstance(SentryMetrics.java:105)
        	at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.initMetrics(SentryPolicyStoreProcessor.java:129)
        	at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.<init>(SentryPolicyStoreProcessor.java:125)
        	at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessorFactory.register(SentryPolicyStoreProcessorFactory.java:31)
        	at org.apache.sentry.service.thrift.SentryService.runServer(SentryService.java:200)
        	at org.apache.sentry.service.thrift.SentryService.call(SentryService.java:167)
        	at org.apache.sentry.service.thrift.SentryService.call(SentryService.java:71)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        	at java.lang.Thread.run(Thread.java:745)
  1.  

Created in 2020 by Google Inc.