The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Starting a RapidMiner Server via AWS

felix_wfelix_w Member Posts: 61 Contributor II
edited November 2019 in Help

Dear Rapidminer Community, 

 

first of all, I am a total noob when it comes to handling servers, sorry ;) There will maybe be some dumb questions in the next paragraphs. ;-) What I would like to do is to set up a Rapidminer Server via AWS and I have a few questions because I am already a little bit confused which steps I have to do to get this server running. (Yes, I have a Rapidminer Server license)

 

I will use the RapidminerServer for collaborative working with 4-6 people, how many GB RAM do I need, how many cores, how many GB space approximately? Is this offer succifient? 16GB RAM, 4 cores https://aws.amazon.com/marketplace/pp/B0742FWHKH . We don't have huge amounts of data atm and will also use our own laptops for calculations etc, the server functions mainly as a place to share processes and some data. That is why an offer, which only makes us pay if we actually use the server would be nice. 

 

I have watched the two videos provided by Rapidminer how to set up Rapidminer Server and just to make sure I would like to summarize the steps here, more or less a to-do list for me: 

 

  1. I have to install Java 8
  2. I have to set up a database, which one would you recommend? MySQL?
    1. I have to install this database on my local machine right?
  3. I would like to use the suggested offer from Rapidminer on AWS (https://aws.amazon.com/marketplace/pp/B0742FWHKH)
  4. Install RapidminerServer software on my laptop
  5. Connect my RapidMiner Studio with my AWS Rapidminer Server through a new repository 

Is this overall correct? 

 

I would use these sources as references for my installation: 

https://rapidminer.com/blog/installing-rapidminer-server-aws/

https://rapidminer.com/blog/installing-rapidminer-server-aws-part-2/

https://rapidminer.com/training/videos/

 

Further questions: 

.) People can work on the server even if my laptop is turned off?  Because if the database is on my laptop and its not turned on, the server should of course be accessible for everybody else.

.) Everybody else who would like to login to the server just needs to create a new repository in his/her RapidMiner Studio and login to the server? Or does everybody have to install RapidMiner Server and Java 8 etc?

.) Will the database on my laptop create a lot of data? 

.) I have RapidMiner Studio 8.1. installed, is this a problem? There is only 8.0 atm available for Server?

 

I know, a lot of questions but I hope you can still help me on my way to installing our own RapidminerStudio. :)

 

Best regards

Felix

Tagged:

Answers

  • felix_wfelix_w Member Posts: 61 Contributor II

    I just found this article: https://docs.rapidminer.com/latest/server/installation/cloud_images.html 

     

    So, if I use the predefined image I don't have to install a database separately? Do I still have to install Java 8 then or does this all come as a bundle in the preset image?

     

    Best regards

    Felix

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    hello @felix_w - no worries about being a noob. We have ALL been there. :)  I did this a while ago and I think (?) it is much easier than all that. The database and the RM Server are all pre-configured on the instance. You literally just choose the size of your instance and press go.

     

    As for the size of the instance, I would recommend starting with a t2.large. It always worked well for me.

     

    Scott

     

     

     

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    oh you asked about your laptop...that does not matter at all. An instance on EC2 is literally a computer that you are renting by the minute in the cloud. It stands by itself. It will be there for however long you leave it on.

     

    Re: Server/Studio versioning. This is VERY important. Server and Studio must be the same version to "talk" to one another. Server and Studio 8.1 were released yesterday.

     

    Re: other users. Once you launch the instance, login as admin and create user accounts. Each user will have her/his own user repo ON THE SERVER (i.e. on that EC2 instance's MySQL database).

     

    Some other pieces of advice:

     

    • The beauty of EC2 is that you can turn on and off your server all the time - and you only pay for when it is ON. Hence mastering the AWS Lambda features is very good - setting CRON triggers to only have your server running when you need it.
    • I would also recommend setting up AWS Lambda scheduled snapshot backups.
    • In general on EC2, storage is pretty cheap; server time is NOT cheap. Turning things on and off is the key.
    • Make sure your security (IAM) is set up so that access to your servers is exactly what you need and nothing more.
    • I would recommend purchasing a dedicated IP address and associating it with your server. It makes accessing your server much easier. Otherwise every time you turn it on and off, you will get a new IP.

    Scott

  • SGolbertSGolbert RapidMiner Certified Analyst, Member Posts: 344 Unicorn

    Just a quick tip: You can set your database in Amazon RDS. You get 20 GB of storage for free and you are moving some of the processing and memory consumption out of your server.

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    absolutely correct. Thanks, @SGolbert. Is the setup with RDS any easier now than it has been in the past?

     

    Scott

     

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    @sgenzer what would be really great are some tips and tricks videos on how to spin up a RM Server on AWS. I did it but it I finally moved to a RM Server on a desktop.

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    are you reading my mind, @Thomas_Ott? I was thinking the exact same thing. :) OK they're on the list.

     

    Scott

     

     

  • felix_wfelix_w Member Posts: 61 Contributor II

    Hi Scott, 

     

    thank you for your detailed reply!

     

    To summarize it for myself, please correct me if I am wrong

    • If I set up a predefined image on AWS I don't have to install a seperate database, Java 8 etc? I only need to decide which instance I take, e.g. t2.large, and AWS sets it up for me? Do I even need to download the RapidMiner Server software?
    • Subsequently, as soon as it is ready I can simply create a server repository in my RapidMiner Studio and connect it to the AWS Rapidminer server as well as everybody else who has access (after I set up their accounts)? That's it?
    • What do you understand by "server time"? Does this mean simply running the server without doing any actual work on it is costly? E.g. the server is online from 8am-8pm but I only do some work from 3pm to 5pm I have to pay the full 12h and not only the 2h I work on it? t2.large costs ~0,09$/h + 0,2$ infrastructure costs = 0,29$/h?
    • Can I turn on and off the server via RapidMiner Server or do I have to do that via my AWS account?
    • Can I configure the IAM and a buy a dedicated IP at AWS or where do I get that?
    • The AWS Lambda feature is accessible via an AWS acocunt?

     

    Best regards

    Felix

  • felix_wfelix_w Member Posts: 61 Contributor II

    Anybody who can answer some of my questions above? :) 

     

    Best regards

    Felix

  • SGolbertSGolbert RapidMiner Certified Analyst, Member Posts: 344 Unicorn

    Hi Felix,

     

    I will try to answer some of the questions, but you will have to contact AWS for IP and billing details.

     

    • The RM Server image on AWS marketplace comes with Java8 and a database (I couldn't find out exactly which one, should be either MySQL or Postgresql). It is ready to go! More info: https://docs.rapidminer.com/latest/server/installation/cloud_images.html
    • T2 instances are charged by the hour. If they are running but not doing calculations, they accumulate "CPU credits". Then if they do heavy calculations later, they go into turbo mode and receive more processing power, but consume CPU credits. If you use the T2 unlimited option, they will always use full processing power when needed but you will get charged for the extra CPU credits that you use.

    Regards,

    Sebastian

  • felix_wfelix_w Member Posts: 61 Contributor II

    Thank you for your answers Sebastian :)

     

    Best regards

    Felix

  • NikouyNikouy Member Posts: 22 Contributor II
    @sgenzer "mastering the AWS Lambda features is very good - setting CRON triggers to only have your server running when you need it."

    Hi Scott,

    How can CRON triggers be integrated with RM and Lambda to spin up an instance only when actually required?

    Thanks



  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    hi @Nikouy try this https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/
Sign In or Register to comment.