QA: RxHadoopMR() job fails with 'Unable to load libhdfs.so' error

  • First make sure that the file 'libjvm.so' is installed on your system:
            sudo find / -mount -name "libjvm*"

            If it shows up as being installed on the system, most likely it is missing from your 'LD_LIBRARY_PATH' variable(see next bullet point).
  • Add the 'libjvm.so' file found on your system to 'LD_LIBRARY_PATH':
            export LD_LIBRARY_PATH=$LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/<path/to>/libjvm.so
  • You may also need to set the environment variables 'HADOOP_HOME' and 'HADOOP_VERSION' on your system:
             These variables should be set as follows(for example):

              HADOOP_HOME=/usr/lib/hadoop/client-0.20

              HADOOP_VERSION=2.0.0-cdh4.4.0
  • Also ensure that the file  '/etc/profile.d/Revolution/bash_profile_additions' exists on all of the nodes of your cluster and has been sourced.
Properties

Article ID: 3104172 - Last Review: Nov 1, 2015 - Revision: 1

Feedback