The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

RM Server 9 running great on Paperspace with GPUs

jacobcybulskijacobcybulski Member, University Professor Posts: 391 Unicorn
edited December 2018 in Help

I have noticed that Paperspace is not featuring strongly in the RM community (in favour of AWS and Azure), however, if you needed to play with deep learning with large GPUs and little expense (e.g. Volta V100 is $2.30 per hour), I can report that the RM Server 9 installs and runs just great on their Ubuntu virtual machines (I have not tested their Windows machines).

 

Things to watch: make sure you install Java 8 (as Java 9 is a default), install Oracle MySQL (use legacy passwords and the current Oracle mysql connector) and avoid Mariadb (unless you want a major cleanup of your machine) and define your Keras and Python settings in [<server>/job-agent/home/config/rapidminer/rapidminer.properties], pointing to the Python in the preinstalled Anaconda with Tensorflow and Keras.

 

As Keras plugin seems memory hungry, increase the container memoryLimit to the max you can (depending on your machine configuration) in [<server>/job-agent/home/config/agent.properties].

 

Last, you can save yourself a lot of time and effort by steering away from bare machines, and instead using some of the pre-configured ML in a Box templates, which they currently have for small GPUs (which I was playing with) and for Volta V100 (in Beta).

 

Have fun -- Jacob

P.S. I found that a container memory limit less than 8Gb causes frequent over the limit warnings, however, Java seemed to recover and garbagge collect the memory to complete a job.

Tagged:

Answers

  • David_ADavid_A Administrator, Moderator, Employee-RapidMiner, RMResearcher, Member Posts: 297 RM Research

    Hi,

     

    thanks for the write-up.

    Have you also tried out to use the new DeepLearning extension from RapidMiner, that supports GPU natively  with the DL4J backend?

    Would be interesting to know how good this works as well.

     

    You can simply switch the backend support between GPU and CPU by setting this server parameter key:

     


    Key
    Values

    		 
    rapidminer.general.nd4j_backend
    		 
    CPU, GPU (choose one)

     

    Best,
    David

  • jacobcybulskijacobcybulski Member, University Professor Posts: 391 Unicorn

    All looks good. I have installed the Deep Learning extension and ran a couple of standard samples on the remote Paperspace server and all worked well. For anything more complex, I'd have to grab the DL4J documentation.

    Jacob

  • David_ADavid_A Administrator, Moderator, Employee-RapidMiner, RMResearcher, Member Posts: 297 RM Research

    This is great to hear.

     

    We like to get more feedback, especially from experiences on different environments.

    Feel free to share any benchmarks you perform.

     

    Best,
    David

Sign In or Register to comment.