The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

problems with the Execute Program operator

kaymankayman Member Posts: 662 Unicorn
edited December 2018 in Help

Going nuts over something that seems pretty trivial.

 

I have a shell script that I want to run from within RM at the end of a workflow. the command I use is as simple as this : /mypath/myshell.sh

Running this shell script directly on my server (Ubuntu 16) works fine, but running it from RM server fails with error message : Process '/mypath/myshell.sh' exited with error code 255.

 

My shell script itself is as shown below, using sftp with a pem file

 

sftp -o ProxyCommand='nc -X connect -x myProxy:myPort %h %p' -i /mypath/mykey.pem myaccount@myDomain <<upload
put /myPath/myFile.csv
quit
upload

Running this using ssh / console works fine, so the syntax used is probably correct.

The only thing I can think about are access rights, my pem file needs to be set to owner read only (600) so if I login as admin and my pem file is assigned to my admin group it works fine, but how does rapidminer communicate with my server? Which group / user account is this using? I assume these are different, and I modified a few times the settings of the pem files (assigning them to root for instance) but no difference.

 

Or is there anything else I might be missing here ?

Or any other advice to get this sh running?

Tagged:

Best Answer

  • kaymankayman Member Posts: 662 Unicorn
    Solution Accepted

    Managed to fix it by 'mixing' the user and group setting, so instead of root:root or user:user I tried user:root, which allowed me to run and add the domain to safe sites when using console. After that it worked fine in RM also.

     

    The joy of Linux :-)

     

    Thanks for the tips !

Answers

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    hi @kayman - yep my money is on access rights as well. Happens to me all the time. Try chmod 700 (I think 600 is R/W only, not execute). 

     

    Scott

     

  • kaymankayman Member Posts: 662 Unicorn

    nope, the problem is that the API I use requires the pem file to be this restrictive. Anything above 600 /different as owner and it just says no. 
    So I would need to know which account RM server is using, if that's the right terminology to use. 

     

    I still have the option to create cron jobs for my shell scripts as an alternative, these do work fine. I would loose the option to start my upload ad hoc but it's not a big problem. 

    Any idea if I can force a saved cron job to start ad hoc from within rapidminer?

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist

    Hey @kayman,

    operator toolbox has a new operator for sftp connection. Maybe this solves your issue?

     

    Best,

    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • kaymankayman Member Posts: 662 Unicorn

    Hi Martin, I noticed this indeed (and some other nice updates to the toolbox btw)

     

    Unfortunatly it does not seem to be able to work with .pem files. I was anyway able to get rid of some of my other sftp scripts that I handled with python by using the new operator, so usefull for sure. 

     

    Now, I'm getting closer to the root cause, my whole ssh flow will probably need to be tailored to root (as this appears the default setting for RM) but I also need to be able to add my sftp site to known hosts while running as root, but since I cannot access my server as root but only as sudo etc etc...

     

    So I'm probably stuck to use cron jobs instead. There are most likely workarounds but my Linux knowledge needs some upgrades also...

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    calling in reinforcements – tagging the RM Server "power team" @jpuente @hhomburg @JessForbesRM @Nils_Woehler


    Scott

     

     

  • jpuentejpuente Employee-RapidMiner, Member Posts: 53 RM Product Management

    Hi kayman,

    RapidMiner Server runs as the user that was chosen during installation. It might be root but, for what you're saying it's probably not in your case. If you can, try logging in as that user and try the script in the command line. That will help to determine the issue.  As Scott and you said, it's probably about permissions. What error appears in Rapidminer?

     

    Jesús

Sign In or Register to comment.