Average

sara20

Hello

For one data set I need label, so I use average of rows for that, now the question is that: why with a simple label the accuracy is very low and why if I use average as an attribute then according to the average I made "UP" and "DOWN" label, the accuracy is 95%?
What is your idea about it?
Is average bring correlation for data? ( I think correlation is not good in data) so why the result is not normal?
Except average what is your suggestion in statistics?

Thank you in advance
sara 

Telcontar120

Hi Sara, your question is not clear to me. Perhaps you could provide a sample process or data file. But in general, if the average contains information from other examples that are not part of your training set then yes you could have information leakage that would bias your model and make it appear stronger than it really is.  

sara20

@Telcontar120

Thank you for the answer. So in this situation how can I make any label for my data? ( instead of average what is your suggestion to choose as a label?)

Sorry the data is not mine and I can not share it.

Regards
Sara

Telcontar120

It's hard to say without knowing anything about the underlying data. But generally I would not create a label by combining information from multiple examples---why not use the attribute itself that you are averaging in its untransformed state as your label?
If you want to predict a classification problem rather than a numerical value, you could also define a threshold value and then create a nominal attribute to specify whether the attribute is above or below that threshold. So, for example, if the value was a sales amount, then you could define a threshold of "high value" transaction at say $1000 and then classify individual examples as either high value or not high value based on that threshold. But you would probably want to consult with a domain expert for the data that you have to determine a threshold like that.

sara20

@Telcontar120

I had the label like the one that you mentioned and again it was useful but very similar to the average because I made an average for one  attribute and less or more than that was my label. So still it brings correlation for the data and the result of accuracy is more than 95%.

Any way thank you for your answer
Sara