The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Rapidminer studio detects pandas module but is unable to load it

VI_DSVI_DS Member Posts: 2 Learner III
edited June 2019 in Help
Hello everyone,

I am trying to set up the python scripting operator and installed Anaconda 3.7. I used the specific python binary to enter the path to python.exe in the base environment.
When I test the path, I get an error message that pandas module is detected but is unable to load.I already reinstalled Anaconda and separately reinstalled the pandas module, still no change in the error message. 
The error log set to "Finest" gives me these messages:

Jun 28, 2019 10:15:41 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder getPathPrefix
FINEST: Python executable C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe detected and it is not a Conda installation.
Jun 28, 2019 10:15:41 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder start
FINEST: Starting Python process C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe using PATH=C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Users\****\AppData\Local\Microsoft\WindowsApps;
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonSetupTester scriptingSetupTest
WARNING: Pandas module detected, but the extension is unable to load it. Check your installation.
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder getPathPrefix
FINEST: Python executable C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe detected and it is not a Conda installation.
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder start
FINEST: Starting Python process C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe using PATH=C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Users\****\AppData\Local\Microsoft\WindowsApps;
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonSetupTester scriptingSetupTest
FINEST: Tested Python executable: C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe

@varunm1
Tagged:

Best Answer

Answers

  • MarlaBotMarlaBot Employee-RapidMiner, Member Posts: 57 Community Manager
    Hi @VI_DS - this is MarlaBot. I found these great videos on our RapidMiner Academy that you may find helpful:
    Instructional Video: Python and PyCharm in RapidMiner (Viewing time: ~5m)
    Instructional Video: The RapidMiner MarketPlace - Extensions (Viewing time: ~5m)
    Instructional Video: hints: Extensions (Viewing time: ~2m)
    Instructional Video: Server-Administration - Queues and Job-Agents: Demo (Viewing time: ~3m)
    Instructional Video: RapidMiner Server Installation - Preparations (Viewing time: ~3m)
    Please LIKE my comment if it helps! 👇

    MarlaBot <3
  • VI_DSVI_DS Member Posts: 2 Learner III
    @hughesfleming68 thanks a lot for the help, that solved the problem. Still would be interesting to know why it works on some machines and not on others
  • hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    edited June 2019
    Hi @VI_DS, I am glad that this got you up and running. I will look into it this weekend. There is a thread about the problem already but I couldn't get the suggested path changes to work on my first try. I will see if I can get it to work and report back.
  • btamasbtamas Employee-RapidMiner, Member Posts: 11 RM Team Member
    Hi @VI_DS ,

    if you want to use Anaconda, you can choose in Settings >> Preferences... >> Python Scripting the conda (anaconda) option as a package manager, then select your desired environment (the base, in your case, which is the default). (If Anaconda is not found, add the main Anaconda and the Anaconda\bin directory to the search paths in the same settings tab first.) This should solve your problem.
    Let me know, if you have still problems with Anaconda 3.7.

    Bence Tamás
  • hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    edited July 2019
    Hi @btamas, unfortunately this does not work in my case. I have to select specific python binaries and  only 3.6.8 works. The conda environment option remains empty on this particular machine that I am using right now even if I add the the correct paths in the 'Search Paths'. I know it can work as I mentioned above but this is a problem that does affect many users. It is still not completely solved and seems specific to Anaconda and Python 3.7.

    https://community.rapidminer.com/discussion/54673/pandas-module-was-not-found
  • phellingerphellinger Employee-RapidMiner, Member Posts: 103 RM Engineering
    Hi All,

    The root cause for this is usually that Conda does not find its DLLs when RapidMiner calls it. Recent Conda versions need a different path setup.

    Python Scripting Extension 9.3.1 brings all kinds of heuristics to avoid any similar issues for Windows Anaconda installations.

    Please let us know if you experience any issue with that version.

    Best,
    Peter 
  • joeanalyticajoeanalytica Member Posts: 7 Contributor II
    Upgrading Rapidminer to 9.4 might solve the problem!
  • hsayhumzahsayhumza Member Posts: 1 Learner III
    edited January 2020
    @joeanalytica
    this did work for me. thank you
  • joeanalyticajoeanalytica Member Posts: 7 Contributor II
    edited January 2020
    hi @hsayhumza .  You're welcome. Great to hear that it worked for you!
  • kaymankayman Member Posts: 662 Unicorn
    @phellinger, I actually just got the same thing after installing the rapidminer python package. Seems it was adding an additional numpy package and I had to remove 2 versions of it. After reinstalling numpy pandas got loaded correctly again.
  • GohGoh Member Posts: 9 Learner I
    I am stuck at an error message when running the Keras Model operator.

    ***
    Unsupported Pandas version
    The current version 1.0.3 of the pandas module for python is not supported.
    Please make sure that the pandas module for Python has at least version 0.12.0.
    ***

    How do I resolve this? How do I get pandas 0.12.0 and install it, if it is still available?
    Are there other workarounds to this problem? I have downgraded my Anaconda from 3.7 to 3.6 in an attempt to fix this. Nonetheless, the problem still exists ...  :/  

  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi @Goh,

    Have you tried to update RapidMiner to the last official version (RapidMiner 9.6) ?

    Regards,

    Lionel
  • GohGoh Member Posts: 9 Learner I
    Yes, I am already using RapidMiner 9.6, downloaded from the official website.
  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    @Goh,

    In this case, uninstall Pandas 1.0.3 and install Pandas 1.0.0.
    If the error still persists, uninstall Pandas 1.0.0 and install Pandas 0.25.3

    Regards,

    Lionel
  • GohGoh Member Posts: 9 Learner I
    I have uninstalled Pandas 1.0.3 and installed 0.25.3.

    The operator before this, Execute Python, worked ok. It could detect the presence of 0.25.3 when I clicked the test button.

    However, the Keras Model operator now displays the error message as:
    The Pandas module for Python was not found.

    How do I solve this?
  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    @Goh,

    It seems to me that the RapidMiner's extension "Keras" is no longer maintained. That's why there is some bugs  like that from my point of view.
     Can you instead : 
     - use the new Deep Learning extension (to install from the MarketPlace)
          OR
     - Build your Keras model manually in a Execute Python operator using Python code.

    Regards,

    Lionel
  • hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    I am not using the Keras extension anymore but I did have it all working with python 3.6.8. You might try creating an environment for that.
  • phellingerphellinger Employee-RapidMiner, Member Posts: 103 RM Engineering
    @Goh

    yes, unfortunately Keras is not maintained the same way as Python Scripting extension is maintained. The latter keeps adapting to the changes the rapidly evolving Python ecosystem dictates.

    My guess is that the cause for the error is the same described above in my comment:
    "The root cause for this is usually that Conda does not find its DLLs when RapidMiner calls it. Recent Conda versions need a different path setup. Python Scripting Extension 9.3.1 brings all kinds of heuristics to avoid any similar issues for Windows Anaconda installations."

    Keras extension calls out to Python, but it does not sets the PATH environment variable the way how Conda requires it to find its dependencies.

    I suggest to follow @lionelderkrikor's recommendations.

    (If you still want to give a last try to the Keras extension and you are using Windows, you may potentially solve it this way: try to add these subfolders into the PATH environment variable before starting RapidMiner: <conda_home>/Library/mingw-w64/bin, <conda_home>/Library/usr/bin, <conda_home>/Library/bin, <conda_home>/Scripts, <conda_home>/bin, where you use the conda install path instead of <conda_home>.)

    Best,
    Peter
Sign In or Register to comment.