stock normalization or prediction (help)

student_compute

Hello
I have the following data

I want to predict the bottom price and the percentage of one share tomorrow
I don't know if this data needs normalization?
And how do I determine for each class?
Someone has the experience to help me
I tried a lot but I can't predict tomorrow's price at RapidMiner
Thankful

student_compute

Hi again
I created a process as follows

For pre-processing and clearing stock data
Is my process and idea right?
What to do after that?
Does anyone know how to help me?
Please ...
Thankful


And
I have the main chart or chart of stock changes. Can we predict the continuation of the chart in the RapidMiner program by entering the stock chart? How????

hughesfleming68

Hello Student Compute, I don't see any missing values in your data and I don't see a need to look for outliers at this point. Isolate your open, high, low and close values and build a standard time series forecasting process. Stock prices usually rise on low volume and fall on high volume so those attributes don't have much predictive value so stick to price. So...if you just have price, should you normalise? Take a look for yourself and see if after normalisation your open, high, low and close values maintain the correct structure.This should answer your question.

The theory works like this. Each stock has its own volatility which can be measured by taking an average of its daily range. Once you know this then you can build some targets that will give you price levels that the stock may reach on the next trading day. Anything beyond these levels could be considered an outlier so you don't need any special outlier detection. In most cases, your time series is going to look like a random walk but that is not necessarily the case. You need to think of it in terms of signal and noise. Sometimes you will have enough signal to get a good prediction and other times you won't.

Any prediction better than random might be considered a good result. Think of it as tossing a weighted a coin over and over again where the distribution of values is not even. This is really the best that you can achieve with financial time series prediction.

Please go back and review how to do basic time series forecasting. This question has been answered by many different people since the beginning of the year so please don't ask again how to do this.

Kind regards,

Alex

Edit...here is an screenshot that will help you understand the idea. No machine learning is really necessary. You would be surprised how often a simple technique like this can capture the range of a financial time series. In many cases, this approach will often get better results than trying to do a regression on the low or high values. 

student_compute

Hello

Yes you are right

I searched a lot. But it's really a training file that can accurately predict a stock's performance. I couldn't find it and couldn't even draw a small sample.

I did a study on stock market analysis, for example candle patterns. But I want to know what can be predicted by machine learning and what can I do in the data mining domain?

I have asked for help many times in the community. But it was just you and one of the friends in the community didn't share their experience and didn't help.

Maybe my expectation was too high.

But I have repeatedly asked the experienced RapidMiner friends to provide an example of a time series in stock price forecasting so that I understand the process. But...

Thank you very much

hughesfleming68

@student_compute, the basics can be seen here. https://www.neuralmarkettrends.com/rapidminer-5-0-video-tutorial-9/. I have sent you these links before.There are also examples that come packaged with Rapidminer. All you have to do is sit down and work through the examples. You have to understand windowing and how to do a sliding window validation. You also have to consider your attributes and that is it. You must understand, if your time series is noise then you are going to predict noise. You won't be able to discover reliable patterns if your data is random.

I think you are being unrealistic. There is no off the shelf process that you can setup in Rapidminer that will predict stock prices with consistent accuracy. Edges in financial markets are very difficult to find. If it was easy, everyone would be doing it and that is not the case. This is an industry that requires specific domain knowledge and a decent understanding of probability. In most cases, price volatility is driven by news flow. Somethings can't be predicted. Trading systems in the real world tend to be reactive rather than predictive. I am not saying that it can't be done but you would have to be ready to be wrong a lot.

Lets say that you were able to build a model that seemed to work. How would you know? You would need a specialised back testing tool and the programming knowledge to build a strategy. This is something you can't do in Rapidminer. Your testing in the end may still be very deceiving. How do you know if you are clever or lucky?

Edit- You have to also take into consideration that the financial world is very secretive. No one with the skill to build profitable predictive models is going to give that information away for free on the internet. You can read hundreds of papers that are written by the academic community that claim to be able to predict financial time series with some degree of accuracy when in fact they are simply curve fitting the past. Don't believe everything that you read. 

sgenzer

@student_compute - writing better questions will likely yield more helpful answers - please read https://community.rapidminer.com/discussion/55505/.

Thank you.

student_compute

Hello
Thank you Mr.@hughesfleming68
I'm based on this link
https://www.youtube.com/watch?v=RtCvcP43C1U
 In Excel using regression I wanted to predict
But the result is .. !!!!!!!!
look
An example of training data

2934
2836
2824
2753
2734
2710
2682
2783
2857
2781
2874
2903
3040
3107
3018
3011
2904
2920
2873
.
.
.

Their regression

y = -17.021x + 3196.5
R² = 0.8638

Test data

1648
1585
1518
1504
1540
1531
1523
1539
1525
1506


And the result predict

-24854.108
-23781.785
-22641.378
-22403.084
-23015.84
-22862.651
-22726.483
-22998.819
-22760.525
-22437.126


I don't know why the results are negative ...!
I did a lot of searching in the forum, Mr. @sgenzer
But I did not find an example that fit my question
I also did a lot of research on YouTube on how regression can predict future stock prices. But I didn't find ....
Anyway, thank you Mr.@hughesfleming68 for his great help
Thanks

hughesfleming68

@student_compute .Take a look at his diagram and duplicate it. This is a good opportunity to understand windowing.

hughesfleming68

The diagram above is the most basic example. Start with univariate forecasting. Adjust your window sizes and experiment with different ML operators. At each step, look to see what happens to your data. Once you understand the basics, then move on to sliding window validation.

As far as using linear regression for stock price prediction..... to make a long story short....that isn't going to work.

Regression problems can be difficult to interpret because depending on your learner, you will have different levels of undershoot or overshoot. You will have to figure out yourself what is acceptable error. Turning the problem into a binary classification problem has many benefits..... the most obvious one being that you are either right or your are wrong.

Edit.. switch off regularization in the GLM. That is a subject for another time.

hughesfleming68

Anyone serious about forecasting needs to look at the results and solutions of the M4 forecasting competitions. https://www.mcompetitions.unic.ac.cy . Worth discussing.

The winner Slawek Smyl from Uber Tech and his solution -

 https://github.com/M4Competition/M4-methods/blob/master/118 - slaweks17/ES_RNN_SlawekSmyl.pdf

student_compute

Hi again Mr. @ hughesfleming68
Thank you so much for taking the time
You are always very helpful in the community.
May I request to send me the xml code of the processed photo?
Thank you for your additional file and tips.
So should I work with a window operator and a neural network?
I try to do as you say. Professor. Wait for the result. Maybe I need your help again.
good day
thank you

hughesfleming68

I would avoid neural nets and work with simpler models. You can get very good results with certain kinds of neural nets but start with models that you can understand. I will upload something for you tomorrow.

student_compute

Thanks

hughesfleming68

Here you go @student_compute. This is just for educational purposes. You will need to use the value series extension. Swap learners and adjust window sizes and see what happens. Learn how to transform the example into a binomial classification.