MapR 4.0.1: Revolution R job fails with segfault

On a MapR 4.0.1 cluster, Revolution R mapreduce jobs may fail with the error:

2015-02-13 08:00:37,657 FATAL [IPC Server handler 2 on 52243] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1423734868663_0289_m_000000_0 - exited : java.io.IOException: R VM exited with exit code: 1, output: 
starting script 
[1] TRUE 
$fileName 
[1] NA 
$start 
[1] 0 
$length 
[1] 0 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libExaCore.so.2(_Z21CriticalSignalHandleri+0x1e)[0x7fe02d3d423e] 
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre/lib/amd64/server/libjvm.so(+0x7e2628)[0x7fe0282f8628] 
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71.x86_64/jre/lib/amd64/server/libjvm.so(JVM_handle_linux_signal+0x95)[0x7fe0282fea55] 
/lib64/libpthread.so.0[0x30cec0f710] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libExaCore.so.2(_ZNK10CxAnalysis19HaveWriteDataSourceEv+0x0)[0x7fe02acbc280] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libExaCore.so.2(_ZN10CxAnalysis18GetWriteDataSourceEv+0x9)[0x7fe02acbcb49] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libRxLink.so.2(_Z19RxComputeHadoopTaskR11RListObjectR11RxAllocator+0x3b3)[0x7fe02e2a5d23] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libRxLink.so.2(_Z10RxUtilBaseP7SEXPREC+0x6db)[0x7fe02e2b2fab] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/library/RevoScaleR/rxLibs/x64/libRxLink.so.2(RxUtil+0x34)[0x7fe02e2b3f14] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/lib/libR.so(+0x9f36b)[0x7fe04083536b] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/lib/libR.so(Rf_eval+0x871)[0x7fe04086be91] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/lib/libR.so(Rf_ReplIteration+0x212)[0x7fe0408946a2] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/lib/libR.so(+0xfea59)[0x7fe040894a59] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/lib/libR.so(run_Rmainloop+0x44)[0x7fe040894f64] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/bin/exec/R(main+0x1b)[0x40084b] 
/lib64/libc.so.6(__libc_start_main+0xfd)[0x30ce41ed5d] 
/usr/lib64/Revo-7.3/R-3.1.1/lib64/R/bin/exec/R[0x400739] 
Segmentation fault.

at com.revolutionanalytics.scaleR.hadoop.Rscript.waitFor(Rscript.java:103) 
at com.revolutionanalytics.scaleR.hadoop.Rscript.run(Rscript.java:59) 
at com.revolutionanalytics.scaleR.hadoop.RDelegatingMapper.run(RDelegatingMapper.java:73) 
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:345) 
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:415) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1469) 
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

On MapR 4.0.1 platforms, this is due to the Revolution R Enterprise environment configuration using the wrong libhdfs.so library.

To work around the problem, confirm the following post-installation configuration steps have been taken for Revolution R 7.3 on MapR 4.0.1 clusters:

On the "client" node, create the native filesystem shared directory for each user:

1. sudo mkdir -p /var/RevoShare/mapr 
3. chmod 777 /var/RevoShare/mapr 
4. add . /usr/lib64/Revo-7.3/scripts/RevoHadoopEnvVars.site to /home/mapr/.bash_profile

On EVERY worker node, make the following changes:

# cp scaleR-hadoop-0.1-SNAPSHOT.jar /opt/mapr/hadoop/hadoop-2.4.1/share/hadoop/common/lib/ 
# ln -s /opt/mapr/hadoop/hadoop-0.20.2/c++/Linux-amd64-64/lib/libhdfs.so /usr/lib64 
# mv /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so /opt/mapr/hadoop/hadoop-2.4.1/lib/native/libhdfs.so.save



The installation of Revolution R automatically creates a link to scaleR-hadoop-0.1-SNAPSHOT.jar in $HADOOP_HOME/lib. However, this is the incorrect directory (a bug has been reported) and the file should be linked or copied in /opt/mapr/hadoop/hadoop-2.4.1/share/hadoop/common/lib/.

Linking /opt/mapr/hadoop/hadoop-0.20.2/c++/Linux-amd64-64/lib/libhdfs.so in /usr/lib64 will ensure that the proper libhdfs.so is found by Revolution R. The third step is necessary to move the problem libhdfs.so out of the way.

Need more help?

Expand your skills
Explore Training
Get new features first
Join Microsoft Insiders

Was this information helpful?

Thank you for your feedback!

Thank you for your feedback! It sounds like it might be helpful to connect you to one of our Office support agents.

×