NoClassDefFoundError Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration

cazjck

Hi,

 I try run Hadoop with single mode on Rapidminer. Howerver, I got issue . I using Rapidminer Studio 7.5

java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.rapidminer.pagerank.utilities.HadoopUtilities.runHadoopLocal(HadoopUtilities.java:173)
	at com.rapidminer.pagerank.operator.PageRankOperator.doWork(PageRankOperator.java:78)
	at com.rapidminer.operator.Operator.execute(Operator.java:1004)
	at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
	at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812)
	at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:1)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807)
	at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428)
	at com.rapidminer.operator.Operator.execute(Operator.java:1004)
	at com.rapidminer.Process.run(Process.java:1315)
	at com.rapidminer.Process.run(Process.java:1179)
	at com.rapidminer.Process.run(Process.java:1132)
	at com.rapidminer.Process.run(Process.java:1127)
	at com.rapidminer.Process.run(Process.java:1117)
	at com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

In build.gradle

dependencies {
	//compile 'com.google.guava:guava:18.0'
	compile "org.apache.hadoop:hadoop-hdfs:2.7.3"
	compile "org.apache.hadoop:hadoop-client:2.7.3"
	compile "org.apache.hadoop:hadoop-mapreduce-client-core:2.7.3"
	compile "org.apache.hadoop:hadoop-yarn-common:2.7.3"
	compile "org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.7.3"
	compile "org.apache.hadoop:hadoop-yarn-client:2.7.3"
	compile "org.apache.hadoop:hadoop-yarn:2.7.3"
	compile "org.apache.hadoop:hadoop-common:2.7.3"
}

Can everyone help me?

cazjck

Help me .

Thomas_Ott

You posted this twice. Doing so will not get your question answered quicker. I will delete the second topic. 

Pavithra_Rao

Hi,

 

I guess if you provide the full path for the yarn.application.classpath property, the NoClassDefFoundError issue gets resolved.

YARN configuration XML could be found here: /etc/hadoop/conf/yarn-site.xml

 

Hope this helps, let us know.

 

Cheers,

cazjck

Thanks you so much.

However,  When I run in Eclipse with Hadoop single mode  && cluster mode , it's ok .

After I install  my extension  , I run extension in Rapidminer, occur exception.

I don't know, How to add classpath  in  extension of Rapidminer ? Can you provide for me one example ?

Sorry, I speak english not good.

Pavithra_Rao

Hi,

 

Could you please check if the Yarn class path is defined in the following XML file:

YARN configuration XML could be found here: /etc/hadoop/conf/yarn-site.xml

user@linux$ sudo vi /etc/hadoop/conf/yarn-site.xml
[ ... ]
<property>
  <name>yarn.application.classpath</name>
    <value>
      $HADOOP_CONF_DIR,
      $HADOOP_COMMON_HOME/*,
      $HADOOP_COMMON_HOME/lib/*,
      $HADOOP_HDFS_HOME/*,
      $HADOOP_HDFS_HOME/lib/*,
      $HADOOP_MAPRED_HOME/*,
      $HADOOP_MAPRED_HOME/lib/*,
      $YARN_HOME/*,
      $YARN_HOME/lib/*,
      /etc/hadoop/conf,
      /usr/lib/hadoop/*,
      /usr/lib/hadoop/lib,
      /usr/lib/hadoop/lib/*,
      /usr/lib/hadoop-hdfs,
      /usr/lib/hadoop-hdfs/*,
      /usr/lib/hadoop-hdfs/lib/*,
      /usr/lib/hadoop-yarn/*,
      /usr/lib/hadoop-yarn/lib/*,
      /usr/lib/hadoop-mapreduce/*,
      /usr/lib/hadoop-mapreduce/lib/*
    </value>
</property>

 Cheers,

cazjck

Thanks you.

But in code initialize Configuration throw exception. You can see some picture below:

 

 

Thomas_Ott

Why don't you just use the Radoop Extension?

cazjck

I want to create a extension using MapReduce, I don't know use Hive and Spark .