How to back-transform differentiated time series data?

manfye

I have done a ARIMA forecasting via a differentiated time series data with an ARIMA (1,1,0), but the data produced by rapidminer is the differentiated data, how do I back transformed it?

Telcontar120

I am not entirely sure what you mean by this--perhaps you can elaborate or attach an example?
@tftemme might be able to help out he has done a lot of the development of the series operators.

lionelderkrikor

Hi @manfye,

You said : 

I have done a ARIMA forecasting via a differentiated time series data with an ARIMA (1,1,0)

Do you mean that you have applied a differentiation method yourself before fitting your data with the ARIMA (1,1,0) model ?
If yes, what differentiation method have you performed on your initial time series ?
 - subtraction
 - logarithm
 - a combination of both ?

I think it is possible to back-transform your predictions by using :  
 -  the "cumsum" (cumulative sum) function in a Python script (in case of subtraction) - I envisage to use Python because I'm not aware of an equivalent function of "cumsum" in Rapidminer.
 - the "exp" function (in case of logarithm)
 - and finally a combination of "cumsum" and "exp" function in the third case above.

To conclude, as Brian said, can you give more detail about your process and your data and ideally share them ?

Regards,

Lionel

lionelderkrikor

Hi again @manfye,

I don't know why, but my intuition tells me that you used the subtraction differentiation method on your original time series.
In this case, a priori you have obtained the predictions on the differentiated data by applying the ARIMA model.
Thus you have obtained data like that : 


Once you have these predictions on the differentiated data, you calculate the cumulative sum on this data and you add to this calculated time series, the first data point of your original time series - in other words, you integrate (from a mathematical point of view) your differentiated data -, and finally you retrieve the original time series with its forecast : 
 

You can adapt the process in attached file to your own data. In my case, I used a time series called "Monthly milk production"
available in RapidMiner repository.

Hope this helps,

Regards,

Lionel

PS : This process is using a Python script, thus, you first have to : 
 - install Python on your computer.
 - install the Python scripting extension from the MarketPlace.
 - set, in RapidMiner settings, the path where your Python.exe file is stored.

tftemme

Hi @manfye

To add on the nice and detailed answer of @lionelderkrikor

ARIMA in itself performs a differentiation before fitting the model to it (the "I" in ARIMA stands for "Integrated"). The number of differentiations applied is specified by the parameter d of the (p,d,q) parameter set. So in your case it was differentiating the input series 1 time and then fitting a p=1, q=0 ARMA model to it. And when applied to perform the forecast the, the ARMA model is used to produce the forecasted values which are then integrated again automatically. 
So in short you don't need to differentiate your input time series by yourself.

Another point. If you still want to integrate a differentiated time series (or any time series by the way), we will have a new Integrate operator in the upcoming 9.7.0 version of RapidMiner Studio. Besides cumulative sum, you can also apply left and right riemann sums and the trapezoidal rule to calculate the integral.

Hopes this helps
Best regards
Fabian

prashant768

Hi @tftemme, I doubt the ARIMA model in RM is doing the inverse transform after applying the differencing parameter. I have tried to perform the same in the data set and the forecasted values are getting too much out of scale. I have attatched the RM process for your reference. 
I am really struggling to configure how to get back the forecasted values into the original scale as I do not want my students to use PYTHON because I am teaching them how to do analysis without any code

hmhsing

Hi @tftemme, I have a similar problem the forecast values are "explosive" when d is not zero. Hope this can be solved, thanks!  https://community.rapidminer.com/discussion/59825/arima-model-forecast-for-nonstationary-data 

rakhuuok

I don't know why, but my intuition tells me that you used the subtraction differentiation method on your original time series.  vidmate.app

stream videos

salmawisoky

Differencing is performed by subtracting the previous observation from the current observation. Inverting the process is required when a prediction must be converted back into the original scale. This process can be reversed by adding the observation at the prior time step to the difference value.