Question about # of logical processor
Learner IIIHi
I would like to find out on the different tier of logical processor in Rapidminer Studio. For eg, if my PC has 8 logical processors, and i am using the free version (which has only 1 logical processor), does it mean that only 1 logical processor on my PC will be used? eg. Rapidminer restricts the usage of the other 3 logical processor
If i purchase the Medium tier of Rapidminer Studio (which has 4 logical processors), only 4 processor will be used during the process?
Next, i would like to find out if the data gets process within my PC or will it be processed on some Rapidminer server? Due to the nature of data privacy and confidentiality, we cannot allow the data to be processed online. So we would like to find out the location where the data gets process.
Appreciate the insights shared.
Regards
Wei Jun
Best Answer
-
rfuentealba
RapidMiner Certified Analyst, Member, University Professor Posts: 568 
Unicorn
Hello Wei Jun,
Let me try to shed some light on this:
RapidMiner Studio is constrained by the amount of rows you can process and the amount of logical processors you can use. RapidMiner Studio doesn't actually restrict the usage of the other logical processors for other processes if that is your concern. The RapidMiner Studio Free license uses the same process it is running to process information, and all your operators must be run sequentially, not in parallel. The RapidMiner Studio Medium license does use the same process + 3 other threads to run operators in parallel if they can be run in parallel.
RapidMiner Studio can work standalone, with no RapidMiner Server installation behind. You can use it on your own desktop, completely disconnected from Internet, while you are sitting down in Pitcairn Island or the Villa Las Estrellas Naval Base in Chile, and it will work. If your computer is powerful but you need even more power, need to serve your information through a network (not necessarily Internet but your local network), schedule tasks, or just create services that consume your data models, you can add a RapidMiner Server but this one will work On Premise, meaning that you need your own server to install it, use it, control it, manage it, etc. Even if you acquire a RapidMiner Server license, your data is yours.
There are two other things that might require explanation:
- Handling a RapidMiner Server is not difficult but requires a certain set of skills. You can use a preconfigured RapidMiner Server on demand. With your very own account you already have RapidMiner Cloud Repository that allows you to upload up to 5 Gb of data there, and run a limited amount of processes. However, you must connect to that instance on the cloud and move your data there to use it, hence it is not trivial to accidentally push some data.
- If you want to have a shared server as part of your Cloud Infrastructure, you can buy RapidMiner On Cloud for a certain amount of credits on Amazon EC2 or Microsoft Azure Platform. Again, you must connect to these instances on the cloud and move your data there to use these, hence it is not trivial to accidentally push some data.
On the other hand, I work at a company that has cybersecurity as part of the core developments (I use RapidMiner for fraud detection and research, among other tasks, so we had to audit this properly). As a result, I found that the only traces that we have seen from our RapidMiner Studio installations and a RapidMiner service are about license updates and what operators you use the most and how, and that helps with the Wisdom Of Crowds, but neither one transmits your data without your consent.
You can test that with mitmproxy and sslsplit if you don't believe me. These are two UNIX tools that allow us to sniff SSL traffic data. You can also use Wireshark but I find it a bit more difficult. Well, the thing is that It's not like we've broken SSL. Therefore, I want to ensure the people at RapidMiner, Inc. and the community that you shouldn't be worried or afraid. Things are looking good for y'all!
All the best,
6
Answers
Hi Rodrigo
Thanks for the detailed explanation
I have one final question though regarding logical processor; If i'm using Rapidminer Medium Edition and my laptop has 8 logical processor, Rapidminer will at most use 4 of the logical processor. The only way to maximum all 8 logical processor on my laptop would be to get the Rapidminer Large Edition?
Regards
Wei Jun
Hi Wei Jun,
That is correct. The only way to use 7-8 cores is buying the RapidMiner Studio Large license. Please notice that the operating system you use and its configuration might reserve one for itself, or will distribute all the tasks evenly depending on the kind of scheduler(s) that the operating system has configured, so you will never be able to dedicate all the cores to your project.
May I ask why is it so important? Even with the software I am building on top of RapidMiner (it will be showcased at RapidMiner Wisdom in New Orleans, BTW! If you are in the United States or want to head there, join us!), I've never required lots of cores to work, but I did require tons of gigabytes of memory.
Perhaps there is a use case I hadn't seen, and would love to know something about it
All the best,
Hi Rodrigo
Wow, really appreciate your time and effort in answering my queries! Nothing special on the logical processors though, i am just curious on how it works for the different tiers cause i cant find much explanation on the web. So i decided to try my luck here xD
Thanks anyways!
Regards
Wei Jun
Hey @18a641r,
we are of course happy to discuss this with you with our professional team! Just write me an email if you want to chat. mschmitz at rapidminer dot com
Cheers,
Martin
Dortmund, Germany
First of all: big thanks to @rfuentealba for the fantastic explanations!!! And please get in touch with @mschmitz to discuss how we can help you.
For the time being, you might also want to check out the white paper below. While you will not see those performance improvements for all situations (like explained before), in certain situations you can get a lot faster process executions by throwing more cores at it :-) Here is the link:
https://rapidminer.com/resource/performance-improvements-rapidminer-studio/
Hope this helps,
Ingo