The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
Difference between result of Rapid miner and Excel removing duplicates function
Learner IIIGood day.
I am new in using RM.
I need to remove duplicates from my dataset within preprocessing step.
SO,
I have 7621 examples as original set.
I used "remove duplicates' function of excel and got 6830 rows ( examples) as a result.
Since, I` m runing the project in RM , I need to clean my data via its operator. Thus, I used "Remove Duplicates operator" , I have choosen "Project name" attribute and run process. As an outcome I got 6854 examples.
My question is why do I have difference between the resulting examples ( 6854 via RM & 6830 via Excel).
I attached my process to this message and asking support for dealing with this problem, please.
Thank you in advance.
Tagged:
0
Answers
Without you providing the dataset, there is no way of knowing for sure. Experiment checking/unchecking the option "Include Special Attributes".
I attached xml file.
The xml file doesn't allow us to actually see the data though, just that you retrieve it as part of your process...
Another thing to test is whether you have any leading or trailing spaces as part of the attribute you are trying to dedupe if it is polynominal. You can use the "Trim" operator in RapidMiner first to make sure it doesn't.
Lindon Ventures
Data Science Consulting from Certified RapidMiner Experts