Ranger Cluster Setup
...
# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
audit_solr_urls=http://localhosthostname:6083/solr/ranger_audits
...
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhosthostname:6083/solr/ranger_audits
...
# DB UserId used for the XASecure schemadb_name=rangerdb_user=rangeradmindb_password=rangeradmin# DB UserId for storing auditlog infromationaudit_db_name=rangeraudit_db_user=rangerloggeraudit_db_password=rangerlogger |
How to Upgrade MySQL 5.1 to MySQL 5.5 on CentOS 6.7 (Optional, only if you install ranger latest versions after 0.7)
1. Confirm your version of MySQL is CentOS 6.7 default
Type in the following to confirm that you have CentOS 6.7 default of MySQL 5.1 installed:
rpm -qa | grep mysql
Outputshouldlooksimilar to the following:
mysql-server-5.1.73-5.el6_6.x86_64mysql-libs-5.1.73-5.el6_6.x86_64mysql-5.1.73-5.el6_6.x86_64
Everything looks good, moving on!
2. Install and activate the REMI and EPEL RPM Repositories
If you have not done so already, install and activate the REMI and EPEL repositories;
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm && rpm -Uvh epel-release-latest-6.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm && rpm -Uvh remi-release-6*.rpm
Now to enable the REMI repository globally:
nano /etc/yum.repos.d/remi.repo
Under the section that looks like [remi] make the following changes:
[remi]name=Remi's RPM repository for Enterprise Linux 6 - $basearch#baseurl=http://rpms.remirepo.net/enterprise/6/remi/$basearch/mirrorlist=http://rpms.remirepo.net/enterprise/6/remi/mirrorenabled=1gpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi
Type CTRL-O to save, and CTRL-X to exit
2. Update MySQL from 5.1 to 5.5
Simply type in the following:
yum -y update mysql*
Once that’s done, we can verify:
rpm -qa | grep mysql
And we should see something similar to the below:
mysql-5.5.45-1.el6.remi.x86_64compat-mysql51-5.1.54-1.el6.remi.x86_64mysql-libs-5.5.45-1.el6.remi.x86_64mysql-server-5.5.45-1.el6.remi.x86_64
Now we need to make sure MySQL is working correctly.
If you’ve set a root password already, type the following;
mysql -u root -p
If you have not yet set a root password, you can simply type;
mysql
You should see something similar to the following
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 1
mysql -u root -p
SET @@global.innodb_large_prefix = 1;
use
mysql_upgrade -u root -p
to solve
SQLException : SQL state: HY000 java.sql.SQLException: Column count of mysql.user is wrong. Expected 42, found 39. Created with MySQL 50173, now running 50556. Please use mysql_upgrade to fix this error. ErrorCode: 1558
error
e. Once all the required properties are updated run the setup.sh script
./setup.sh |
Will encountering the following error, if one user ranger >0.7.0 version and with a sql version <5.5:
then run
SET @@global.innodb_large_prefix = 1;
Error executing: CREATE TABLE `x_portal_user` ( `id` bigint(20) NOT NULL AUTO_INCREMENT, `create_time` datetime DEFAULT NULL, `update_time` datetime DEFAULT NULL, `added_by_id` bigint(20) DEFAULT NULL, `upd_by_id` bigint(20) DEFAULT NULL, `first_name` varchar(1022) DEFAULT NULL, `last_name` varchar(1022) DEFAULT NULL, `pub_scr_name` varchar(2048) DEFAULT NULL, `login_id` varchar(767) DEFAULT NULL, `password` varchar(512) NOT NULL, `email` varchar(512) DEFAULT NULL, `status` int(11) NOT NULL DEFAULT '0', `user_src` int(11) NOT NULL DEFAULT '0', `notes` varchar(4000) DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `x_portal_user_UK_login_id` (`login_id`), UNIQUE KEY `x_portal_user_UK_email` (`email`), KEY `x_portal_user_FK_added_by_id` (`added_by_id`), KEY `x_portal_user_FK_upd_by_id` (`upd_by_id`), KEY `x_portal_user_cr_time` (`create_time`), KEY `x_portal_user_up_time` (`update_time`), KEY `x_portal_user_name` (`first_name`(767)), KEY `x_portal_user_email` (`email`), CONSTRAINT `x_portal_user_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`), CONSTRAINT `x_portal_user_FK_upd_by_id` FOREIGN KEY (`upd_by_id`) REFERENCES `x_portal_user` (`id`) )AUTO_INCREMENT=2 ROW_FORMAT=DYNAMIC;
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes ErrorCode: 1071
2017-06-17 01:02:03,297 [E] xa_core_db.sql file import failed!
2017-06-17 01:02:03,297 [I] CORE_DB_SCHEMA might being imported by some other process
2017-06-17 01:04:03,382 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "show tables like 'xa_access_audit';"
2017-06-17 01:04:03,673 [I] Table xa_access_audit does not exist in database ranger
2017-06-17 01:04:03,673 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='CORE_DB_SCHEMA' and active='N' and updated_by='ranger24355-1000.dev.continuuity.net';"
2017-06-17 01:04:03,955 [E] CORE_DB_SCHEMA import failed!
...
sudo chgrp ranger /var/log/ranger-usersync
cd /usr/local
tar zxvf ~/root/dev/incubator-ranger/target/ranger-0.6.3-usersync.tar.gz
...
sudo ln -s ranger-0.6.3-usersync ranger-usersync
inside /usr/local cd ranger-usersync
POLICY_MGR_URL = http://localhosthostname:6080SYNC_SOURCE = unix logdir = /var/log/ranger/usersyncexport JAVA_HOME=/usr/lib/jvm/java
sudo ln -s ranger-0.6.3-usersync ranger-usersync
cd /usr/local/ranger-0.6.3-usersync
...
./ranger-usersync-services.sh start
make sure this script run without error and there is no symlink loop.
Enabling Ranger HDFS Plugins (Optional, from HDFS, HIVE etc choose one plugin) (must choose one, suggest hive)
...
scp /Users/chaoranwang/Desktop/cdap-security-extn/cdap-ranger/lookup/target/lookup-0.1.0-SNAPSHOT-jar-with-dependencies.jar rangerhostname:~
ssh into rangerhostname
then cp lookup jars to, if not there use mkdir -p to make path:
cp /homeusr/chaoranlocal/lookup-0ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/cdap
cp /home/chaoran/lookup-0.1.0-SNAPSHOT-jar-with-dependencies.jar /usr/local/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins
chown ranger:ranger to this jar filethis jar file
chown ranger:ranger cdap
cd cdap
mv /usr/local/cdap-ranger/lookup-0.1.0-SNAPSHOT.jar ./
chown ranger:ranger /usr/local/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/lookup-0.1.0-SNAPSHOT-jar-with-dependencies.jar
ls -l to look at user/group property to confirm it has ranger:ranger
...
scp /Users/chaoranwang/Desktop/cdap-security-extn/cdap-ranger/binding/target/binding-0.1.0-SNAPSHOT.jar cdaphostname:~cp binding jar from cdap server
to mkdir -p /usr/local/cdap-ranger/
cp /home/chaoran/binding-0.1.0-SNAPSHOT.jar /usr/local/cdap-ranger/
chown cdap:cdap to this jar filecdap /usr/local/cdap-ranger/binding-0.1.0-SNAPSHOT.jar
and then restart server
restart cdap service:
...
tail -f /usr/local/ranger-admin/ews/logs/catalina.out
correctly setup should show:
Ranger plugin and CDAP architecture
Make enforcement working
configure cdap property: first 3
...