How to run Revolution R Enterprise in mixed-mode Hadoop environments

Revolution R Enterprise may be run in a mixed-mode Hadoop environments. That is, the ability to run Revolution R jobs using Yarn when the default cluster job mode is "classic" MR1. To configure Revolution R to run in this manner:

a. Temporarily set the cluster default mode to Yarn.

b. Rebuild the RevoHadoopEnvVars.site file to include environment for Yarn:
# cd /usr/lib64/Revo-7.4/scripts
# ./getHadoopEnvVars.py
c. Edit /usr/lib64/Revo-7.4/scripts/RevoHadoopEnvVars.site and change
HADOOP_CMD=hadoop
to
HADOOP_CMD=hadoop2
d. Change the default cluster mode to classic.

Note that specifying the job queue in the Hadoop compute context in the script should use the Yarn format (mapreduce.jobs.queuename) but using the deprecated MR1 format (mapred.jobs.queue.name) may be tolerated.
Eigenschappen

Artikel-id: 3104125 - Laatst bijgewerkt: 31 okt. 2015 - Revisie: 1

Feedback