Setting up Apache Sentry from source on a Coopr CDH cluster
Background
Since no released version of CDH supports the version of Apache Sentry that CDAP is integrating with, we cannot use Cloudera Manager to set up Sentry for testing. This page lists the steps to build Sentry from source and start the on a CDH cluster created using Coopr.
Prereqs
- git
- mvn
Setting up MySQL (Optional)
Create User
mysql> CREATE USER 'sentry'@'localhost' IDENTIFIED BY 'sentry'; Query OK, 0 rows affected (0.00 sec) mysql> CREATE DATABASE sentry_metastore; Query OK, 1 row affected (0.00 sec) mysql> GRANT ALL ON sentry_metastore.* TO 'sentry'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye
Verify
# mysql -usentry -hlocalhost -psentry Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 17 Server version: 5.1.73-log Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use sentry_metastore; Database changed
Steps
- Clone Apache Sentry source from https://github.com/apache/incubator-sentry
- There is a problem compiling the sentry-tests module right now, so comment it out temporarily from the root
pom.xml
until we find a fix mvn clean install -DskipTests
will generate a sentry distribution undersentry-dist/target/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/
. Only takes a few mins.cd sentry-dist/target/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/apache-sentry-1.7.0-incubating-SNAPSHOT-bin/
Create a
sentry-site.xml
in the conf directory with the following contents. This is a bare-minimum sentry-site.xml without kerberos.<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>sentry.service.security.mode</name> <value>none</value> </property> <property> <name>sentry.service.admin.group</name> <value>admin1</value> </property> <property> <name>sentry.service.allow.connect</name> <value>impala,hive</value> </property> <property> <name>sentry.store.jdbc.url</name> <value>jdbc:derby:;databaseName=metastore_db;create=true</value> </property> <property> <name>sentry.store.jdbc.driver</name> <value>org.apache.derby.jdbc.EmbeddedDriver</value> </property> <property> <name>sentry.store.jdbc.password</name> <value>sentry</value> </property> </configuration>
Set up the sentry policy store using a Derby DB
bin/sentry --command schema-tool --conffile conf/sentry-site.xml --dbType derby --initSchema
Or, for MySQL:
bin/sentry --command schema-tool --conffile conf/sentry-site.xml --dbType mysql --initSchema --verbose
Note: mysql jdbc driver jar should be in sentry/lib. Also, sentry does not accept a user without password for mysql (and perhaps postgresql etc)
Start the sentry service
bin/sentry --command service --conffile conf/sentry-site.xml &> sentry-service.log &
If you get the following error, it is because of an old
metrics-core.jar
in thehadoop classpath
. If you see this, remove the file/usr/lib/hadoop-mapreduce/metrics-core-3.0.2.jar
until we fix the error, and restart sentry using the above step.Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.NoSuchMethodError: com.codahale.metrics.JmxAttributeGauge.<init>(Ljavax/management/MBeanServerConnection;Ljavax/management/ObjectName;Ljava/lang/String;)V at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at org.apache.sentry.service.thrift.SentryService.waitOnFuture(SentryService.java:311) at org.apache.sentry.service.thrift.SentryService$CommandImpl.run(SentryService.java:397) at org.apache.sentry.SentryMain.main(SentryMain.java:114) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.lang.NoSuchMethodError: com.codahale.metrics.JmxAttributeGauge.<init>(Ljavax/management/MBeanServerConnection;Ljavax/management/ObjectName;Ljava/lang/String;)V at com.codahale.metrics.jvm.BufferPoolMetricSet.getMetrics(BufferPoolMetricSet.java:45) at org.apache.sentry.provider.db.service.thrift.SentryMetrics.registerMetricSet(SentryMetrics.java:149) at org.apache.sentry.provider.db.service.thrift.SentryMetrics.<init>(SentryMetrics.java:97) at org.apache.sentry.provider.db.service.thrift.SentryMetrics.getInstance(SentryMetrics.java:105) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.initMetrics(SentryPolicyStoreProcessor.java:129) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.<init>(SentryPolicyStoreProcessor.java:125) at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessorFactory.register(SentryPolicyStoreProcessorFactory.java:31) at org.apache.sentry.service.thrift.SentryService.runServer(SentryService.java:200) at org.apache.sentry.service.thrift.SentryService.call(SentryService.java:167) at org.apache.sentry.service.thrift.SentryService.call(SentryService.java:71) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
Related content
Created in 2020 by Google Inc.