"Error Updating Statistics"

sgenzer

I think this is the day when RM decides to throw me all sorts of weird errors.  I have been getting this one many times over the years (this is not new): "Error executing background job 'Updating Statistics': java.lang.NullPointerException":  

I always click Close and move on with my day (clicking "Report Bug" just sends me to the Community forum...)

But today RM has decided to get serious and not execute the Statistics tab at all!  Here's what I get now:

 

Tried restarting several times, etc...

 

Help?

Scott

 

sgenzer

ok I have a hunch it has to do with nominal or text attributes with "?" in them.  Perhaps RM is confused because that's the way it indicates missing values as well?

Thomas_Ott

I have no idea, haven't seen this error before. Can you attach the log files. I'll ping the Developers. 

sgenzer

hmm can you remind me where the log files are located on a mac?    Here's what the log tab says:

 

May 24, 2017 7:56:21 AM WARNING: Cannot calculate statistics COUNT for attribute From-User: no value given...

 

I think the .ioo file was corrupted somehow.  Can't quite figure it out.  I'm attaching it here (nothing confidential - it's a twitter feed).

 

Scott

kypexin

Hi @sgenzer

 

By any chance, hasn't you got a clue on the solution of this problem yet?

 

I keep getting the very same error mostly after joining 2 datasets, and it also results in a failure to store the same data set in a Server repository reporting Error 500 (though it can be stored in a local repo). For testing purposes I tried removing duplicates, removing double attributes, materializing data, renaming by generic names, but nothing helps. 

MartinLiebig

Hi,

 

can you please post this on support.rapidminer.com so our professional support team can look into it?

 

Cheers,

Martin

kypexin

---

@mschmitz wrote:

Hi,

can you please post this on support.rapidminer.com so our professional support team can look into it?

 

---

Done! -> Case #4526

Thanks Martin.

kypexin

Hi again @sgenzer

 

In the meanwhile, can you still tell me more about the data and process? It happens that I am struggling with exactly the same issue, but there are also some specific things to mention: 

 

1. Initial dataset is fetched from PostgreSQL and then stored as an ExampleSet in a RapidMiner SERVER repository. 
2. Then I try to fetch it, do some filtering and store it back also in SERVER repo. 

If I store it in a local repo, no problems occur. So I suppose at least in my case this is something to do with data transmission between studio and server for some reason. 

 

What kind of setup do you yes and where the data comes from?

sgenzer

Good morning @kypexin - interesting as I had posted this thread in May and have not seen that issue in a while.  I assume you're running 7.6.1?  I'm running Studio 7.6.1 on a Mac Pro (3.5 GHz 6-Core Intel Xeon E5 w 64GB 1866 MHz DDR3 RAM) and macOS Sierra 10.12.6.

 

Scott

 

kypexin

Hi @sgenzer - mine is Studio 7.6.001 run on WIndows virtual machine with RDP + Server 7.6 run on Linux virtual machine. 

 

Currently the only workaround for me is to store examplesets in a local repository, this helps to avoid the problem. Transferring data between studio and server sometimes causes the named error, it involves nominal text fields but I am not able to locate the cause more precisely. Still waiting for some solution from support guys...

sgenzer

Hmm ok.  Let's get this going again and get this solved.  Can you post process and log files so we can easily reproduce?

Scott

 

kypexin

Hi Scott, 

 

I would need to take some time to prepare that, as the data contains personal data and cannot be shared strainght away. I guess I still would try to make a reproducible setup in a short while. 

 

Thanks. 

sgenzer

I completely understand.  Whenever you can, please let me know.  Thanks as always for the feedback.

Scott

kypexin

Hi @sgenzer

 

I have struggled quite long with the problem, and now I am 95% sure that the cause of the issue was in the initial data which contained Western European characters (originally data comes from Spanish customers). Most likely this caused problems in conjunction with moving the data between local Studio and remote Server repository. So as a fast workaround I just get rid of all Western European characters, replacing them to Latin, and this solved the problem for now. 

sgenzer

got it.  OK thanks @kypexin.  Yes I don't know if it's just me but I am encountering more and more weird encoding issues these days.  I feel your pain.  Not much we can do except learn how to deal with weird old data.  Unless some clever person can develop a nice predictive model that will choose the encoding for us? 

 

Scott

 

sgenzer

m_alexakis