Installing LLAP on a Secured Cluster
Review the prerequisites for installing LLAP on a secured cluster before you begin.
- Ensure that the Hive user exists on each node, and configure the following: - Create local directories that are similar to those set up for the yarn.nodemanager.local-dirs property: - mkdir -p /grid/0/hadoop/llap/local chown -R hive /grid/0/hadoop/llap 
- On the "setup" node, ensure that the Hive user has access to its HDFS home directory: - hadoop fs -mkdir -p /user/hive hadoop fs -chown -R hive /user/hive r 
 
- Set up keytabs. - You can perform this step on the "setup" machine and distribute it to the cluster, or you can perform this step on each node. The following example shows how to perform this step on each node. Use kadmin.local if under root; otherwise, use kadmin. 
- On each node (specified by their fully qualified domain names), create the host and headless principals, and a keytab with each: - kadmin.local -q 'addprinc -randkey hive@EXAMPLE.COM' kadmin.local -q "addprinc -randkey hive/<fqdn>@EXAMPLE.COM" kadmin.local -q 'cpw -pw hive hive' kadmin.local -q "xst -norandkey -k /etc/security/keytabs/hive.keytab hive/<fqdn>@EXAMPLE.COM" kadmin.local -q "xst -norandkey -k /etc/security/keytabs/hive.keytab hive@EXAMPLE.COM" chown hive /etc/security/keytabs/hive.keytab 
- On the "setup" node, create and install, as the Hive user, the headless keytab for Slider: - kadmin.local -q "xst -norandkey -k hive.headless.keytab hive@EXAMPLE.COM" chown hive hive.headless.keytab kinit -kt /etc/security/keytabs/hive.keytab hive@EXAMPLE.COM slider install-keytab --keytab hive.headless.keytab --folder hive --overwrite 
- If you want to use web UI SSL, set up the keystore for SSL. - Note that Keystore is often set up for other web UIs: for example HiveServer2. If the keystore is not already set up, perform the following steps: - Create the Certificate Authority (CA). - On the setup node, create the CA parameters: - cat > /tmp/cainput << EOF US California Palo Alto Example Certificate Authority Certificate Authority example.com . EOF 
- Create the CA: ![[Note]](../common/images/admon/note.png) - Note - The JAVA_HOME must be set. The default Java truststore password must be changed. - mkdir -p /etc/security/certs/ openssl genrsa -out /etc/security/certs/ca.key 4096 cat /tmp/cainput | openssl req -new -x509 -days 36525 -key /etc/security/certs/ca.key \ -out /etc/security/certs/ca.crt echo 01 > /etc/security/certs/ca.srl echo 01 > /etc/security/certs/ca.ser keytool -importcert -noprompt -alias example-ca –keystore \ $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -file \ /etc/security/certs/ca.crt rm /tmp/cainput 
 
- Create the certificate. - On the "setup" node, create the keystore parameters. In the following example, llap00 is the password specified for the new keystore: - hostname -f > /tmp/keyinput hostname -d >> /tmp/keyinput cat >> /tmp/keyinput << EOF Example Corp Palo Alto CA US yes llap00 llap00 EOF 
- Generate a keystore, a certificate request, and a certificate, and then import the certificate into the keystore: - cat /tmp/keyinput | keytool -genkey -alias hive -keyalg RSA -keystore \ /etc/security/certs/keystore.jks -keysize 4096 -validity 36525 -storepass llap00 keytool -certreq -alias hive -keystore /etc/security/certs/keystore.jks \ -storepass llap00 -file /etc/security/certs/server.csr openssl x509 -req -days 36525 -in /etc/security/certs/server.csr \ -CA /etc/security/certs/ca.crt -CAkey /etc/security/certs/ca.key \ -CAserial /etc/security/certs/ca.ser -out /etc/security/certs/server.crt keytool -import -alias hive -keystore /etc/security/certs/keystore.jks \ -storepass llap00 -trustcacerts -file /etc/security/certs/server.crt chown hive:hadoop /etc/security/certs/keystore.jks /etc/security/certs/server.crt chmod 640 /etc/security/certs/keystore.jks /etc/security/certs/server.crt rm /tmp/keyinput 
- Distribute the keystore and certificate to each node: - On each node, create the directory: - mkdir -p /etc/security/certs 
- Upload the files from the "setup" node: - scp … /etc/security/certs/* …@node:/etc/security/certs/ 
- Import the CA: - chown hive:hadoop /etc/security/certs/* chmod 640 /etc/security/certs/keystore.jks /etc/security/certs/server.crt keytool -importcert -noprompt -alias example-ca -keystore \ $JAVA_HOME/jre/lib/security/cacerts -storepass changeit -file \ /etc/security/certs/ca.crt 
 
- Configure LLAP and generate the package. - Specify the following properties in the - /etc/hive/conf/hive-site.xmlfile:- Table 9.6. Properties to Set in hive-site.xml for Secured Clusters - Property - Values - hive.llap.daemon.work.dirs - hive.llap.daemon.work.dirs - hive.llap.daemon.keytab.file - hive.llap.daemon.keytab.file - hive.llap.daemon.service.principal - hive.llap.daemon.service.principal - hive.llap.daemon.service.ssl - True - hive.llap.zk.sm.principal - hive@EXAMPLE.COM - hive.llap.zk.sm.keytab.file - /etc/security/keytabs/hive.keytab - hive.llap.zk.sm.connectionString - ZooKeeper connection string: for example, <machine:port,machine:port, ...> - hadoop.security.authentication - kerberos - hadoop.security.authorization - true - Following is an example of these properties set in the llap-daemon-site.xml file: - <property> <name>hive.llap.daemon.work.dirs</name> <value>/grid/0/hadoop/llap/local</value> </property> <property> <name>hive.llap.daemon.keytab.file</name> <value>/etc/security/keytabs/hive.keytab</value> </property> <property> <name>hive.llap.daemon.service.principal</name> <value>hive/_HOST@EXAMPLE.COM</value> </property> <property> <name>hive.llap.daemon.service.ssl</name> <value>true</value> </property> <property> <name>hive.llap.zk.sm.principal</name> <value>hive@EXAMPLE.COM</value> </property> <property> <name>hive.llap.zk.sm.keytab.file</name> <value>>/etc/security/keytabs/hive.keytab</value> </property> <property> <name>hive.llap.zk.sm.connectionString</name> <value>127.0.0.1:2181,128.0.0.1:2181,129.0.0.1:2181</value> </property> <property> <name>hadoop.security.authentication</name> <value>kerberos</value> </property> <property> <name>hadoop.security.authorization</name> <value>true</value> </property>- Optionally, you can also use hive.llap.daemon.acl and hive.llap.management.acl to restrict access to the LLAP daemon protocols. - The Hive user must have access to both. 
 - Specify the following properties in the - ssl-server.xmlfile.- Ensure that you perform this step before you create the LLAP package. - Table 9.7. Properties to Set in ssl-server.xml for LLAP on Secured Clusters - Property - Values - ssl.server.truststore.location - Path to Java truststore: for example, /jre/lib/security/cacerts - ssl.server.keystore.location - /etc/security/certs/keystore.jks - ssl.server.truststore.password - changeit Note: This is the default password. - ssl.server.keystore.password - llap00 - ssl.server.keystore.keypassword - llap00 - Following is an example of these properties set in the - ssl-server.xmlfile:- <property> <name>ssl.server.truststore.location</name> <value>/jre/lib/security/cacerts</value> </property> <property> <name>ssl.server.keystore.location</name> <value>/etc/security/certs/keystore.jks</value> </property> <property> <name>ssl.server.truststore.password</name> <value>strong_password</value> </property> <property> <name>ssl.server.keystore.password</name> <value>llap00</value> </property> <property> <name>ssl.server.keystore.keypassword</name> <value>llap00</value> </property>- Generate the LLAP package. ![[Important]](../common/images/admon/important.png) - Important - Ensure that JAVA_HOME and HADOOP_HOME are set before generating the package. JAVA_HOME and the site.global.library_path property in the appConfig.json configuration file are set using JAVA_HOME and HADOOP_HOME. If you see problems such as a missing native library, check the appConfig.json configuration file. - Make sure that LLAP package generation is done under the hive user because some HDFS paths in the configuration are user-specific. You can modify the paths after package generation 
- To generate the LLAP package, run the following command, setting parameters as described in the LLAP Package Parameters table: - hive --service llap --name <llap_svc_name> --instances <number_of_cluster_nodes> --cache <cache_size>m --xmx <heap_size>m --size ((<cache_size>+<heap_size>)*1.05)m --executors <number_of_cores> --loglevel <WARN|INFO> --args " -XX:+UseG1GC -XX:+ResizeTLAB -XX:+UseNUMA -XX:-ResizePLAB" - Table 9.8. LLAP Package Parameters - Parameter - Recomended Value (based on daemons using all available node resources) - --instances <cache_size> - Set to number of cluster nodes that you to use for LLAP. - --cache <cache_size> - <YARN_maximum_container_size> - (<hive.tez.container.size> * <number_of_cores>) - <hive.tez.container.size> is the setting for this property found in the hive-site.xml file. Depending on the size of the node, specify a minimum of 1-4 GB. - --xmx <heap_size> - For medium-sized nodes: - <hive.tez.container.size> * <number_of_cores> * (0.8 to 0.95) - Where <hive.tez.container.size> is the setting for this property found in the hive-site.xml file. - Ensure that the setting for --xmx is 1GB less than (<hive.tez.container.size> * <number_of_cores>). - For smaller nodes: - Use the same formula as for medium-sized nodes, but multiply by 0.8 - --executors <number_of_cores> - Set to the number of CPU cores available on nodes running NodeManager. Set this value even if CPU scheduling is enabled in YARN. - Set the --loglevel parameter to INFO when you are troubleshooting or testing. The INFO option provides verbose output. In a production environment, set the --loglevel parameter to WARN, which only outputs a message to the logs if there is a warning or error. This makes the logs easier to read and reduces load on the node. ![[Note]](../common/images/admon/note.png) - Note - The recommended values listed in the LLAP Package Parameters table represent a sample configuration. LLAP also can be configured to use a fraction of node resources. 
- Specify the keytab settings in the slider-appmaster section of the appConfig.json configuration file if they have not already been specified: - "components": { "slider-appmaster": { … existing settings … "slider.hdfs.keytab.dir": ".slider/keytabs/llap", "slider.am.login.keytab.name": "hive.headless.keytab", "slider.keytab.principal.name": "hive@EXAMPLE.COM"
 

