The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
"Performance and normalization"
Contributor I
I have question regarding normalization of data and impact on performance (prediction).
I normalize sample attribute values (Z-transfromation, Portion or Range) and than use slidingwindow validation (x,1,1,1) with OneR classifier.
Normalization positively improves performence but I´m not sure if it is correct and normalization of whole dataset somehow project future values to the past .
Question: can I apply normalization on whole sample (dataset) in case of slidingwindow?
ExampleSet
Normalization ? (Z-Transformation)
SlidingWindow (window,1,1)
OneR
ModelApplier
Performance
I normalize sample attribute values (Z-transfromation, Portion or Range) and than use slidingwindow validation (x,1,1,1) with OneR classifier.
Normalization positively improves performence but I´m not sure if it is correct and normalization of whole dataset somehow project future values to the past .
Question: can I apply normalization on whole sample (dataset) in case of slidingwindow?
ExampleSet
Normalization ? (Z-Transformation)
SlidingWindow (window,1,1)
OneR
ModelApplier
Performance
Tagged:
0
Answers
I think you're right, the normalisation, because it runs over all the examples, is data-snooping, and it is quite a fiddle to get it right ( you have to repeatedly make/save/apply the normalisation model - a quick search on this forum will find some code ). On the other hand, well done for spotting the danger; I take it you've already searched this forum on sliding window validation - I've warned elsewhere of the data-snooping risks it entails.