The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
Split the dataset
ikayunida123
Member Posts: 17 
Contributor II
Hello everyone!
So I'm doing a text classification right now. And I want to ask how to split the dataset into data training and data testing on Rapidminer. I know there are some operators like Split Data or Split Validation, but looks like it's splitting the data automatically(?) So I don't know which one is data training or which one is data testing.
My teacher wants me to compare the result of text classification that I'm doing manually and the result of my RapidMiner process. So I must make sure the data training or data testing in those two processes are same.
Please help me. Thank you :catvery-happy:
Tagged:
0
Answers
Hi @ikayunida123
It depends on how you chose your data for training or testing.
Let's say you have 10 samples from 1 to 10, and you train your model manually with samples 1, 2 and 3, and test your model with 4 to 10, you should choose "linear sampling" on both "Split Data" and when configuring the "Split Validation". If, however, you chose specifically (e.g.) examples 1, 4, and 6 to train your model and the rest to test it, you might prefer not to work with split data but by creating two datasets that are equal to your choices, and build the model without split validation.
There are three ways to create data samples in RapidMiner (well, there are four but the fourth one is "automatically choose between stratified or shuffled depending on the data types you have") : "Linear" is 1, 2, 3... "Shuffled" is random, and "Stratified" is shuffled but trying to maintain the proportions between your training data and your testing data.
Regarding Split Data, Split Validation and DIY validation, I can post you some pictures on what is the case for each one:
Split Validation:
Split Data:
With DIY Validation, data splitting is your responsibility. Basically, your model looks much like the "Split Data" model, except that you have two Retrieve operators, one with your chosen training data and other with your chosen testing data. TBH I was too lazy to build an example.
For the sake of completion, I prefer to do Cross Validation whenever I have enough memory and processor to use it (or use it as a pretext to ask my boss to buy more memory and a better processor). It is exactly the same as the Split Validation, but let's say you have 100 examples and you want to part them in 5 folds, you have an iterator: use examples 1 to 20 for testing and the rest for training, then examples 21 to 40 for testing, then 41 to 60 and so on... More folds means smaller examples but more iterations. There is also the "leave one out" option but with this option enabled the amount of computing power required is... quite high if your dataset is large.
Hope this helps,
Rodrigo.
Hello Mr. @rfuentealba :catvery-happy: Your answer definitely makes me understand the concept of validation in RapidMiner. Thank you for your help! But the one that exactly close to my needed (from your explanation) is DIY validation. Can you give me an example of DIY validation? Any pictures or XML process is okay. Thank you and have a nice day :catvery-happy:
You can use the Log operator to capture whatever intermediate results you want from the training and the testing data inside a split validation. Just make sure you use the local random seed option so your results are reproducible.
Lindon Ventures
Data Science Consulting from Certified RapidMiner Experts
Hi @ikayunida123,
If I understood it correctly, you picked up some data by hand, and you are performing calculations manually, and your teacher wants you to see how performance differs between what you do on paper and pencil (May Odin bless your patience for such a task, I failed COBOL twice at the Uni because I didn't have enough of it). So you want something that can help you define that you want exactly that data for training and that other data for testing, am I right? In that case, the Log operator that @Telcontar120 suggested might not work. (However, that's a good catch! You might want to try the other way round: run your process on RapidMiner and use the Log operator to see how your training data looks like, and then take paper and pencil with that data, that's much less hassle!)
Well, DIY means "Do It Yourself" (I had a joke with "Just Do It" but I just didn't... For reference, that is a slogan for a known sportswear company), and it means getting rid of all the help given to you by RapidMiner process blocks.
The first thing you have to get rid of is our good friend the Split Data operator, and the examples must be split by yourself. Then, apply everything as you would, but... before applying the Performance block, you have to define which columns in your result have label and prediction roles. The Performance operator takes these two to analyze how far was your algorithm from the truth, and that's transparent to you when using the Split Validation or the Split Data methods.
Finally, apply Performance as normal, et voilá: DIY Validation! Here is a screenshot.
Cheers and have a nice weekend,
Hello @Telcontar120 thank you for your suggestion :cathappy:
But I'm still didn't understand. Can you please give me an example of how to use the log operator in my case? I'm sorry for asking too much questions.
This's my process :
Thank you.
Here is an example process which uses split validation to create a model and logs the train performance and the text performance separately.
Lindon Ventures
Data Science Consulting from Certified RapidMiner Experts