The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
best unbiased estimator vs second moment
Maven
Hello
A quite unimportant question:
Is there any specific reason why you use the second moment instead of the best unbiased estimator for variance ? I have often stepped over different calculations in different libaries, which sometimes caused a little headache "why the results differ now ?".
greetings
Steffen
A quite unimportant question:
Is there any specific reason why you use the second moment instead of the best unbiased estimator for variance ? I have often stepped over different calculations in different libaries, which sometimes caused a little headache "why the results differ now ?".
greetings
Steffen
0
Answers
this question somehow seems to be like a question of taste. As far as I know, there is no particular reason, why RM uses the second moment rather than the best unbiased estimator for variance. But moreover, this exclusive statement is not true for all RM variance/standard deviation calculations. Hence, I think we should discuss this topic internally, maybe centralize the aforementioned calculations and clearly communicate which way we chose to calculate and possibly also why. Maybe our Wiki will be a good place to explain and document such calculations for users and developers. What do you think?
Regards,
Tobias
Ah yeah, the wiki thing. I was contemplating a lot over some kind of "Data Mining Wiki". As far as I see, there is no such thing as a successful Data Mining Wiki out there, ignoring Wikipedia, which already contains a lot of articles about statistics and various algorithms.
I looked at your wiki, too and thought about writing something. To be honest: This are my main problems...
- The wiki is proprietary: If the situation of your company changes in the future (good or bad),there is always the chance someone will pull the plug.
- The money thing: I am still amazed about RapidMiner and I wish you (truly) success with your courses and the enterprise edition (also I am not able to afford it (yet :P)). Knowledge is money (it is as it is), so maybe you are not going to be too amazed about a well written and somehow complete wiki.
So, these are my problems.I am somehow disappointed about the community of RapidMiner. A lot of people jump in to ask questions, but only few are trying to answer or to participate generally (jump by for a little talk for example). This may not be obvious at the moment, but it was in the old sourceforge forum. So the question raises: WHO is going to write articles for the wiki ?
I hope you understand
greetings
Steffen
PS: Where is Jean Charles ??? ;D
I am here and...oh gosh, I was to write on the wiki !! ::) ;D
Well, to say it clearly, I began to write a few sketches, for instance "regexp". I had the idea to write two types of articles :
- "as is" operator description, that is to reproduce the operator tree and each "F1 box" text
- cross-theme articles, as "regexp", "attributes' roles", each of these articles giving a stub towards more specific operators...
You will see that I began with the following articles :
http://yale.sourceforge.net/wiki/index.php/Regular_expressions
http://yale.sourceforge.net/wiki/index.php/Mapping_the_data_in_RapidMiner_table
http://yale.sourceforge.net/wiki/index.php/Category:Data_Formats
http://yale.sourceforge.net/wiki/index.php/Computing_load_issues
What do you think ? any idea of "cross-theme" articles ? "Encoding" ? "XPath" ? "relational algebra" ? "Handling ExampleSource" ? "text mining basics" ?
At the moment, I am sure that it is just needed to "cast the sketch and keywords" for each foreseen article...
Cheers,
Jean-Charles.
Regarding the community aspect: Two people for the wiki.
More to come ?
Please dont get me wrong. I am more the pessimistic type of guy, the advocatus diaboli. I like to contribute, I just want to assure (and motivate myself), that I do not spend my precious time in vain.
greetings
Steffen
PS: Nice to see you
just a few thoughts: Well, this is basically true for almost all wikis on the net. If Wikipedia decides to do so, there isn't much you could do about that. And chances are probably much better for a Wiki for an open-source software than for most proprietary products. To be honest: there is no reason for us to pull the plug but we would be really interested to see a larger community ourself: independently of the large amounts of valuable wisdom which could be written in a Wiki I am pretty sure that we still have a lot of knowledge about Data Mining in general and RapidMiner in particular in our brains. And that's what people are paying us for, not the information which is available freely in the net anyway. So you can be assured: the Wiki, the freely available written tutorial, RapidMiner itself: everything will stay free and open and will remain on the new (although it might move like this forum for example).
We don't have a problem with this as long as it helps to let the community grow and helps other users. As I said: I am sure that we still know even more than the things which could and probably would be written and we could provide this additional amount of knowledge together with our experience to our customers.
And here I fully agree: there only few people who step by on a regular base. But Data Mining is a highly specialized field and only few people work on it. There are also (related) communities like statistics and also communities built around other products so I find it quite natural that the group of "core users" grows only slowly. On the other hand, it was really great to see how people like you, Steffen and Jean-Charles but also others of course started to form a small user community. And we highly appreciate that, not only since your answers always are of really high quality.
And there other movements, too, for example a first user group meeting in New York. We will certainly organize a free 1-day workshop for users where we can meet and exchange our experiences. So in my understanding, there are a lot of users who just have a problem and want an answer - but there is also a small group of people who start to form a really community around of software. And this actually is one of the greatest things for us here at Rapid-I.
Ok, back to the Wiki. We at Rapid-I will certainly also provide some articles - at least for the most common questions we often have to answer here in the forum (and there also a lot of people who mail us directly although we cannot guarantee an answer for those support requests).
But some people simply have to start and others will follow as the information grows. I also do not think that there is a good data mining wiki around in the net but we could start to build it and other people will follow then. So we would provide a free Wiki and a free tool to use the processes and hints given there. I strongly believe that all sides will benefit from that (I would have founded a company around an open-source software otherwise). But of course among these side there always will be people who will not contribute but only benefit. But that's ok to me.
Cheers,
Ingo
[qoute]
And we highly appreciate that, not only since your answers always are of really high quality.
[/quote]
thank you
Well, that was quite motivating. I wont promise anything, cause I am afraid of a lie, but ....
good night
Steffen
I just want to remember how long it took to set up this forum...
All the best,
Ingo
About the wiki, especially about "operators" pages : I would like to suggest a kind of form, which would take the following structure :
[tt]==Name==
==Group==
==Class==
==Input==
==Output==
==Min Inner==
==Max Inner==
==Inner operator conditions==
==Description==[/tt]
It reproduces the "F1" box, but is there any "template" facility for reusing such a structure ?
Cheers,
Jean-Charles.
good idea. But I have no idea if a template mechanism exists for Wikis. But maybe we can just post the operator template somewhere on the main page so that others can simply copy it from there.
Cheers,
Ingo