The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
Simulator and test-training data in Automodel
Chemical_eng
Member Posts: 16 Contributor II
in Help
Hello,
I am using AutoModel. I have some questions :
1. Is the simulator based on test data, training data or all data ?
2. How do I ensure that my test dataset is balanced, I have a lot of categorical variables, how do I ensure then the test dataset is balanced ?
3. Can I see metrics for both training and test error rates ? I think I only see for test.
Thanks
I am using AutoModel. I have some questions :
1. Is the simulator based on test data, training data or all data ?
2. How do I ensure that my test dataset is balanced, I have a lot of categorical variables, how do I ensure then the test dataset is balanced ?
3. Can I see metrics for both training and test error rates ? I think I only see for test.
Thanks
Tagged:
0
Best Answers
-
MartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data ScientistHi @Chemical_eng ,1. Simulator uses mostly none of the two data sets. It is in the end applying the model on the data set you configure. The data is only used to determine min and max.2. Usually you won't balance on attributes, but only on labels? You can do this by changing the costs/gains matrix3. Only test error rates are reported, because train error rates are rarely of any use.BR,Martin- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany0 -
MartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data ScientistHi @Chemical_eng ,i suppose not all random seeds are set to a fixed seed, so that the split and also the randomness in some algorithms (or the randomness introduced by parallel computation) does change the results slightly.The results should not differ much, right?BR,Martin- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany0
Answers
3. I wanted the error on train dataset to compare to test for overfitting/underfitting purposes, but is ok.
Also another question : 1. I am getting different result when I run train the algorithm in the same dataset, I assume is because of random errors by the algorithm parameters or by choosing the data for train-test split. I am using the model for optimization and I see it gives me different recommendations every time I train, any ideas on how to keep the model fixed ? can we fix these random parameters or choose and average or best combination?