...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
ranger cluster setup
Services
Add
What cluster to build
1.Cdap single node
For Services add
Spark-historyserver
Hive-server2
Generic-users
500 GB
2x large
2.A separate ranger-server on any system with centOS6
create a 4.2 cdap-single node
git checkout tag release-ranger-0.7.1
git checkout tags/release-ranger-0.7.1
How to install ranger on a cluster:
How to install maven on the cluster to build
Step by step
2 (don't do)
install ranger 0.6.3 instead
First use root
sudo su -
Install Maven
...
Install gcc
yum install gcc Clone the ranger source code
mkdir ~/devcd ~/devgit clone https://github.com/apache/incubator-ranger.gitcd incubator-ranger...
Build the source
cd ~/dev/incubator-ranger
...
ranger-0.5.0-admin.tar.gz ranger-0.5.0-kms.tar.gz ranger-0.5.0-storm-plugin.tar.gz ranger-0.5.0-hbase-plugin.tar.gz ranger-0.5.0-knox-plugin.tar.gz ranger-0.5.0-usersync.tar.gz ranger-0.5.0-hdfs-plugin.tar.gz ranger-0.5.0-migration-util.tar.gz ranger-0.5.0-yarn-plugin.tar.gz ranger-0.5.0-hive-plugin.tar.gz ranger-0.5.0-solr-plugin.tar.gz ranger-0.5.0-kafka-plugin.tar.gz ranger-0.5.0-src.tar.gzInstall Steps for Ranger Policy Admin on RHEL/CentOS
cd /usr/local
tar zxvf ~/dev/incubator-ranger/target/ranger-0.6.3-admin.tar.gz
ln -s ranger-0.6.3-admin ranger-admin
cd ranger-admin
...
Add these:
```
db_root_user=root
db_root_password= <enter_your_password>
```
Install and configure Solr or SolrCloud
cd ~/dev/incubator-ranger/security-admin/contrib/solr_for_audit_setup
...
Run the ./setup.sh script as root to install Solr. This will create install notes at
> /opt/solr/ranger_audit_server/install_notes.txt
./setup.sh
Start/Stop and Common Info about Solr or SolrCloud
```
Start and Stoping Solr:
Login as user solr or root and the run the below commands to start or stop Solr:
...
# DB UserId used for the XASecure schemadb_name=rangerdb_user=rangeradmindb_password=rangeradmin# DB UserId for storing auditlog infromationaudit_db_name=rangeraudit_db_user=rangerloggeraudit_db_password=rangerlogger |
How to Upgrade MySQL 5.1 to MySQL 5.5 on CentOS 6.7 (Optional, only if you install ranger latest versions after 0.7)
1. Confirm your version of MySQL is CentOS 6.7 default
Type in the following to confirm that you have CentOS 6.7 default of MySQL 5.1 installed:
rpm -qa | grep mysql
Outputshouldlooksimilar to the following:
mysql-server-5.1.73-5.el6_6.x86_64mysql-libs-5.1.73-5.el6_6.x86_64mysql-5.1.73-5.el6_6.x86_64
Everything looks good, moving on!
2. Install and activate the REMI and EPEL RPM Repositories
If you have not done so already, install and activate the REMI and EPEL repositories;
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm && rpm -Uvh epel-release-latest-6.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm && rpm -Uvh remi-release-6*.rpm
Now to enable the REMI repository globally:
nano /etc/yum.repos.d/remi.repo
Under the section that looks like [remi] make the following changes:
[remi]name=Remi's RPM repository for Enterprise Linux 6 - $basearch#baseurl=http://rpms.remirepo.net/enterprise/6/remi/$basearch/mirrorlist=http://rpms.remirepo.net/enterprise/6/remi/mirrorenabled=1gpgcheck=1gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi
Type CTRL-O to save, and CTRL-X to exit
2. Update MySQL from 5.1 to 5.5
Simply type in the following:
yum -y update mysql*
Once that’s done, we can verify:
rpm -qa | grep mysql
And we should see something similar to the below:
mysql-5.5.45-1.el6.remi.x86_64compat-mysql51-5.1.54-1.el6.remi.x86_64mysql-libs-5.5.45-1.el6.remi.x86_64mysql-server-5.5.45-1.el6.remi.x86_64
Now we need to make sure MySQL is working correctly.
If you’ve set a root password already, type the following;
mysql -u root -p
If you have not yet set a root password, you can simply type;
mysql
You should see something similar to the following
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 1
mysql -u root -p
SET @@global.innodb_large_prefix = 1;
use
mysql_upgrade -u root -p
to solve
SQLException : SQL state: HY000 java.sql.SQLException: Column count of mysql.user is wrong. Expected 42, found 39. Created with MySQL 50173, now running 50556. Please use mysql_upgrade to fix this error. ErrorCode: 1558
error
e. Once all the required properties are updated run the setup.sh script
./setup.sh |
Encountering the following error:
then run
SET @@global.innodb_large_prefix = 1;
Error executing: CREATE TABLE `x_portal_user` ( `id` bigint(20) NOT NULL AUTO_INCREMENT, `create_time` datetime DEFAULT NULL, `update_time` datetime DEFAULT NULL, `added_by_id` bigint(20) DEFAULT NULL, `upd_by_id` bigint(20) DEFAULT NULL, `first_name` varchar(1022) DEFAULT NULL, `last_name` varchar(1022) DEFAULT NULL, `pub_scr_name` varchar(2048) DEFAULT NULL, `login_id` varchar(767) DEFAULT NULL, `password` varchar(512) NOT NULL, `email` varchar(512) DEFAULT NULL, `status` int(11) NOT NULL DEFAULT '0', `user_src` int(11) NOT NULL DEFAULT '0', `notes` varchar(4000) DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `x_portal_user_UK_login_id` (`login_id`), UNIQUE KEY `x_portal_user_UK_email` (`email`), KEY `x_portal_user_FK_added_by_id` (`added_by_id`), KEY `x_portal_user_FK_upd_by_id` (`upd_by_id`), KEY `x_portal_user_cr_time` (`create_time`), KEY `x_portal_user_up_time` (`update_time`), KEY `x_portal_user_name` (`first_name`(767)), KEY `x_portal_user_email` (`email`), CONSTRAINT `x_portal_user_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`), CONSTRAINT `x_portal_user_FK_upd_by_id` FOREIGN KEY (`upd_by_id`) REFERENCES `x_portal_user` (`id`) )AUTO_INCREMENT=2 ROW_FORMAT=DYNAMIC;
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes ErrorCode: 1071
2017-06-17 01:02:03,297 [E] xa_core_db.sql file import failed!
2017-06-17 01:02:03,297 [I] CORE_DB_SCHEMA might being imported by some other process
2017-06-17 01:04:03,382 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "show tables like 'xa_access_audit';"
2017-06-17 01:04:03,673 [I] Table xa_access_audit does not exist in database ranger
2017-06-17 01:04:03,673 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='CORE_DB_SCHEMA' and active='N' and updated_by='ranger24355-1000.dev.continuuity.net';"
2017-06-17 01:04:03,955 [E] CORE_DB_SCHEMA import failed!
...
https://www.if-not-true-then-false.com/2010/install-mysql-on-fedora-centos-red-hat-rhel/
Installing the Ranger UserSync Process (must)
mkdir /data/logs/ranger-usersync
...
./ranger-usersync-services.sh start
Enabling Ranger HDFS Plugins (Optional, from HDFS, HIVE etc choose one plugin) (must choose one, suggest hive)
cd /usr/local
tar zxvf ~/dev/incubator-ranger/target/ranger-0.6.3-hdfs-plugin.tar.gz
...
./enable-hdfs-plugin.sh
Restart all hadoop services in:
/etc/init.d/hadoop-hdfs-datanode restart
...
How to configure Ranger and CDAP to work together.
Install HIVE:
/usr/hdp/2.3.4.7-4/hadoop
...
less /var/log/cdap/master-cdap-ranger063v24563-1000.dev.continuuity.net.log
ranger-admin restart
Process for a ranger plugin code change
0. Build code: mvn clean package -DskipTests
...