The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Java error with Image Handling and Deep Learning extensions

yho67yho67 Member Posts: 4 Learner II
edited April 2022 in Help
Hello everyone!

I reach to you because I get an error with java when trying to to a DeepLearning process

1. Overview of the problem

I wanted to breach the topic of image handling, so I went onto the marketplace and got the three extensions Image Handling 0.2.001, Deep Learning 1.2.001, and ND4J Back End 1.2.000

However, after reading the image correctly, I get an error when reaching the Deep Learning (tensor) module. When running for the first time I get:
Exception: java.lang.UnsatisfiedLinkError
Message: no jniopencv_core in java.library.path

When re-running any time after the first time, I get :
Exception: java.lang.NoClassDefFoundError
Message: Could not initialize class org.bytedeco.javacv.OpenCVFrameConverter$ToMat

I am working with the image set MNIST (sorry can't post the link on my message, but the standard image dataset) 
I am on Windows 10, using RapidMiner9.10.

2. More details on my process and the error

Here is the XML for my very simple process

<?xml version="1.0" encoding="UTF-8"?><process version="9.10.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="9.10.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="image_handling:read_image_meta_data" compatibility="0.2.001" expanded="true" height="68" name="Read Image Meta Data" width="90" x="45" y="85">
        <parameter key="directory" value="C:/Users/yho67/Desktop/data_science/samy_lesson/MNIST/trainingSet"/>
        <parameter key="use_label" value="true"/>
      </operator>
      <operator activated="true" class="image_handling:image_pre_processor" compatibility="0.2.001" expanded="true" height="103" name="Pre-Process Images" width="90" x="179" y="85">
        <parameter key="path" value="Path"/>
        <parameter key="use_label" value="false"/>
        <process expanded="true">
          <connect from_port="transform" to_port="transform"/>
          <portSpacing port="source_transform" spacing="0"/>
          <portSpacing port="sink_transform" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="deeplearning:dl4j_tensor_sequential_neural_network" compatibility="1.2.001" expanded="true" height="145" name="Deep Learning (Tensor)" width="90" x="313" y="85">
        <parameter key="epochs" value="10"/>
        <parameter key="use_miniBatch" value="false"/>
        <parameter key="batch_size" value="32"/>
        <parameter key="log_each_epoch" value="true"/>
        <parameter key="epochs_per_log" value="10"/>
        <parameter key="optimization_method" value="Stochastic Gradient Descent"/>
        <parameter key="backpropagation" value="Standard"/>
        <parameter key="backpropagation_length" value="50"/>
        <parameter key="use_early_stopping" value="false"/>
        <parameter key="condition_strategy" value="score improvement"/>
        <parameter key="patience" value="5"/>
        <parameter key="minimal_score_improvement" value="0.0"/>
        <parameter key="best_epoch_score" value="0.01"/>
        <parameter key="max_iteration_score" value="3.0"/>
        <parameter key="max_iteration_time" value="10"/>
        <parameter key="updater" value="Adam"/>
        <parameter key="learning_rate" value="0.01"/>
        <parameter key="momentum" value="0.9"/>
        <parameter key="rho" value="0.95"/>
        <parameter key="epsilon" value="1.0E-6"/>
        <parameter key="beta1" value="0.9"/>
        <parameter key="beta2" value="0.999"/>
        <parameter key="RMSdecay" value="0.95"/>
        <parameter key="weight_initialization" value="Normal"/>
        <parameter key="bias_initialization" value="0.0"/>
        <parameter key="use_regularization" value="false"/>
        <parameter key="l1_strength" value="0.1"/>
        <parameter key="l2_strength" value="0.1"/>
        <parameter key="cudnn_algo_mode" value="Prefer fastest"/>
        <parameter key="infer_input_shape" value="true"/>
        <parameter key="network_type" value="Simple Neural Network"/>
        <parameter key="use_local_random_seed" value="false"/>
        <parameter key="local_random_seed" value="1992"/>
        <process expanded="true">
          <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Fully-Connected Layer" width="90" x="112" y="34">
            <parameter key="neurons" value="100"/>
            <parameter key="activation_function" value="ReLU (Rectified Linear Unit)"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Fully-Connected Layer (2)" width="90" x="313" y="34">
            <parameter key="neurons" value="100"/>
            <parameter key="activation_function" value="ReLU (Rectified Linear Unit)"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <operator activated="true" class="deeplearning:dl4j_output_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Output Layer" width="90" x="514" y="34">
            <parameter key="output_type" value="FullyConnected"/>
            <parameter key="loss_function" value="Multiclass Cross Entropy (Classification)"/>
            <parameter key="neurons" value="1"/>
            <parameter key="activation_function" value="Softmax"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <connect from_port="in layerArchitecture" to_op="Add Fully-Connected Layer" to_port="layerArchitecture"/>
          <connect from_op="Add Fully-Connected Layer" from_port="layerArchitecture" to_op="Add Fully-Connected Layer (2)" to_port="layerArchitecture"/>
          <connect from_op="Add Fully-Connected Layer (2)" from_port="layerArchitecture" to_op="Add Output Layer" to_port="layerArchitecture"/>
          <connect from_op="Add Output Layer" from_port="layerArchitecture" to_port="out layerArchitecture"/>
          <portSpacing port="source_in layerArchitecture" spacing="0"/>
          <portSpacing port="sink_out layerArchitecture" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="image_handling:read_image_meta_data" compatibility="0.2.001" expanded="true" height="68" name="Read Image Meta Data (2)" width="90" x="45" y="289">
        <parameter key="directory" value="C:/Users/yho67/Desktop/data_science/samy_lesson/MNIST/testSet"/>
        <parameter key="use_label" value="true"/>
      </operator>
      <operator activated="true" class="image_handling:image_pre_processor" compatibility="0.2.001" expanded="true" height="103" name="Pre-Process Images (2)" width="90" x="179" y="289">
        <parameter key="path" value="Path"/>
        <parameter key="use_label" value="false"/>
        <process expanded="true">
          <connect from_port="transform" to_port="transform"/>
          <portSpacing port="source_transform" spacing="0"/>
          <portSpacing port="sink_transform" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="nd4j:apply_model_generic" compatibility="1.2.000" expanded="true" height="82" name="Apply Model (Generic)" width="90" x="380" y="340"/>
      <connect from_op="Read Image Meta Data" from_port="output" to_op="Pre-Process Images" to_port="example set"/>
      <connect from_op="Pre-Process Images" from_port="tensor" to_op="Deep Learning (Tensor)" to_port="training set"/>
      <connect from_op="Deep Learning (Tensor)" from_port="model" to_op="Apply Model (Generic)" to_port="model"/>
      <connect from_op="Read Image Meta Data (2)" from_port="output" to_op="Pre-Process Images (2)" to_port="example set"/>
      <connect from_op="Pre-Process Images (2)" from_port="tensor" to_op="Apply Model (Generic)" to_port="unlabelled data"/>
      <connect from_op="Apply Model (Generic)" from_port="labelled data" to_port="result 2"/>
      <connect from_op="Apply Model (Generic)" from_port="model" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

As for the error code, the first time I run the process I get :

Exception: java.lang.UnsatisfiedLinkError
Message: no jniopencv_core in java.library.path
Stack trace:

  java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
  java.lang.Runtime.loadLibrary0(Runtime.java:871)
  java.lang.System.loadLibrary(System.java:1124)
  org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1718)
  org.bytedeco.javacpp.Loader.load(Loader.java:1328)
  org.bytedeco.javacpp.Loader.load(Loader.java:1132)
  org.bytedeco.opencv.global.opencv_core.<clinit>(opencv_core.java:16)
  java.lang.Class.forName0(Native Method)
  java.lang.Class.forName(Class.java:348)
  org.bytedeco.javacpp.Loader.load(Loader.java:1200)
  org.bytedeco.javacpp.Loader.load(Loader.java:1148)
  org.bytedeco.javacv.OpenCVFrameConverter.<clinit>(OpenCVFrameConverter.java:43)
  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)
  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)
  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)
  java.security.AccessController.doPrivileged(Native Method)
  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)
  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.Process.executeRoot(Process.java:1464)
  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)
  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)
  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

Cause
Exception: java.lang.UnsatisfiedLinkError
Message: C:\Users\yho67\.javacpp\cache\dl-windows-libs-1.2.1-all.jar\org\bytedeco\opencv\windows-x86_64\jniopencv_core.dll: Can't find dependent libraries
Stack trace:

  java.lang.ClassLoader$NativeLibrary.load(Native Method)
  java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
  java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
  java.lang.Runtime.load0(Runtime.java:810)
  java.lang.System.load(System.java:1088)
  org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1668)
  org.bytedeco.javacpp.Loader.load(Loader.java:1328)
  org.bytedeco.javacpp.Loader.load(Loader.java:1132)
  org.bytedeco.opencv.global.opencv_core.<clinit>(opencv_core.java:16)
  java.lang.Class.forName0(Native Method)
  java.lang.Class.forName(Class.java:348)
  org.bytedeco.javacpp.Loader.load(Loader.java:1200)
  org.bytedeco.javacpp.Loader.load(Loader.java:1148)
  org.bytedeco.javacv.OpenCVFrameConverter.<clinit>(OpenCVFrameConverter.java:43)
  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)
  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)
  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)
  java.security.AccessController.doPrivileged(Native Method)
  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)
  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.Process.executeRoot(Process.java:1464)
  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)
  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)
  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

 
And any run after the first, I get this error message :

Exception: java.lang.NoClassDefFoundError
Message: Could not initialize class org.bytedeco.javacv.OpenCVFrameConverter$ToMat
Stack trace:

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)
  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)
  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)
  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)
  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)
  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)
  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)
  java.security.AccessController.doPrivileged(Native Method)
  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)
  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)
  com.rapidminer.operator.Operator.execute(Operator.java:1023)
  com.rapidminer.Process.executeRoot(Process.java:1464)
  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)
  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)
  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

Sorry for the long message, I didn't know exactly what part would be useful, so I included a lot of informations in the end.
If anybody could help me, that would be amazing !

Cheers,
Victor S.
Tagged:

Best Answer

  • yho67yho67 Member Posts: 4 Learner II
    Solution Accepted
    Hi David,

    So, after your explanation, we tried different things, and managed to find the problem ! It was resolved by installing Visual C++.

    My apologies to have bothered you while it was actually something missing from my side... :( 

    If anyone else ever has this trouble, I hope this solution will help !

    Cheers,
    Victor S.

Answers

  • David_ADavid_A Administrator, Moderator, Employee-RapidMiner, RMResearcher, Member Posts: 297 RM Research
    Hi Victor,

    thanks for reaching out and the very detailed error description. That's always helpful.
    We'll have a look at the issue.

    Best,
    David
  • yho67yho67 Member Posts: 4 Learner II
    Hi David,

    Thanks for you help ! In the meantime, I tried reinstallingRapidMiner, but well, it didn't change anything ^^"

    Cheers,
    Victor S.
  • MateMate Employee-RapidMiner, Member Posts: 14 RM Team Member
    edited April 2022
    Hey Victor,
    could you please attach your logs ?
    ./RapidMiner/rapidminer-studio.log

    (in the future, please do use the "Attach file" feature :)) 

    Cheers,
    Mate
  • David_ADavid_A Administrator, Moderator, Employee-RapidMiner, RMResearcher, Member Posts: 297 RM Research
    Thanks for the update.

    I've tried to reproduce the error, but failed so far.

    Do you try to train the model using a GPU (under Settings/Preferences -> Backend). If so, does it work, if you run the process with CPU as backend.

    Can you execute other DL processes? Perhaps one of the tutorial processes?

    Best,
    David
  • yho67yho67 Member Posts: 4 Learner II
    Hey Mate, David,

    Thank you for your time !

    I attached my log when trying to launch the process 3 times in a Row. Sorry, I didn't see this option on my first message, my mistake ...

    Another information I've come to : I'm executing this process for a lesson on Machine Learning, and after asking around my fellow students, knowing we did the same things at the same time give or take a week (installing RapidMiner at the same time, downloading extension at approximately the same time etc.), some had no trouble and a minority had exactly the same error messages than I had. I couldn't figure out the common link though. (So yeah, not too useful, sorry ...)

    @David, I didn't touch the settings, so they are still on default option for the Backend, which after checking is CPU-OpenBLAS. I actually don't have a GPU on my Laptop.

    For other process:
    - the Tutorial Process: "MNIST classification gives me the exact same errors".
    - the Tutorial Process : "Creating a Deep Learning Model on multivariate time series data" worked fine actually (I just tried after reading your message, sorry for not giving you this information sooner!)

    If you need anything else from me that could be relevant, please let me know. Once again, thanks a lot for the time you're taking for that !

    Cheers,
    Victor S.
  • David_ADavid_A Administrator, Moderator, Employee-RapidMiner, RMResearcher, Member Posts: 297 RM Research
    Hi Victor,

    thanks again for the detailed feedback.
    Unfortunately it looks, like everything from our side works as intended and the error is somehow related to your (and your fellow students) Windows installation.

    You might need to search how to fix the opencv error regarding the missing jniopencv_core.dll.


    Best,
    David


Sign In or Register to comment.