Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

aruberutou

Hello,

Thanks for producing such a wonderful tool. It has helped my work tremendously.

One minor niggle, however, frustrated me for a quite a bit and I wanted to share my work-around.

I have been using SentiWordNet on an OSX installation, and it tended to die once it hit a certain number of files. Now, I am not sure if its a coding issue (i.e., should it really be opening so many files at once), but it easily fixed by adding "-XX:-MaxFDLimit" to the RapidMiner-Studio-App.sh file, at the appropriate places, as shown below:

# launch RapidMiner Studio
if [ $# -gt 0 ]; then
eval \"$JAVA_HOME\"/bin/java $JVM_OPTIONS -XX:-MaxFDLimit -cp \"${RM_CLASSPATH}\" com.rapidminer.launcher.GUILauncher \"$@\"
else
eval \"$JAVA_HOME\"/bin/java $JVM_OPTIONS -XX:-MaxFDLimit -cp \"${RM_CLASSPATH}\" com.rapidminer.launcher.GUILauncher

Again, I am not sure if it is this way by design, but maybe the RM team should consider including this option by default in the OSX app. The Windows version doesn't seem to suffer from this limitation.

Thanks,

Marco_Boeck

Hi,

thanks for bringing this to our attention! This looks more like a bug in the SentiWordNet extension to me, though. I have created an internal ticket for it.
We won't globally disable the JVM open file limit of a little more than 10.000 files because that could potentially do more harm than good.

Regards,
Marco

aruberutou

Hi,

Thanks for the quick follow-up. I haven't noticed any issues since disabling the limit; however, I am sure it is probably there for a reason.

weewey

I have this problem as well. RM always give me an error when doing sentiment analysis to more than 200 over tweets.

I tried the solution that aruberutou mentioned but I'm still facing the problem. /   ??? ???

This is the error message:
"Error reading /usr/local/WordNet- 3.0/dict:/usr/local/WordNet-3.0/dict/index.verb (Too many open files)"

aruberutou

Hi, weewey,

Sorry for the late response. In order to resolve this, I also had to increase the respective limits in OSX as well.

Please refer to the following websites for details:

http://blog.mact.me/2014/10/22/yosemite-upgrade-changes-open-file-limit
http://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite/828010#828010

weewey

@aruberutou

Awesome! Thanks alot!! I got it fixed already.

weewey

@aruberutou

I realise that rapidminer server result in the same problem too.

Do you know how to resolve it?

Appreciate your help

Marco_Boeck

Hi,

depending on your OS, it's either the standalone.conf (Linux/Mac) or the standalone.conf.bat file which you can find in the SERVER/bin folder. Look for a line starting with "JAVA_OPTS=" and add your parameters to the existing options.

Regards,
Marco

rachel_lomasky

The problem is that you have put the "Open WordNet Dictionary" operator inside of the "Process Documents" operator.  You should open it outside, and then use a remember with the type "WordNet Dictionary."

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="wordnet:open_wordnet_dictionary" compatibility="5.3.000" expanded="true" height="68" name="Open WordNet Dictionary" width="90" x="380" y="340">
<parameter key="resource_type" value="directory"/>
<parameter key="directory" value="%{wordnet_directory}"/>
</operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="remember" compatibility="7.4.000" expanded="true" height="68" name="Remember" width="90" x="581" y="340">
<parameter key="name" value="wordnet"/>
<parameter key="io_object" value="WordnetDictionary"/>
<parameter key="store_which" value="1"/>
<parameter key="remove_from_process" value="true"/>
</operator>
</process>

 

And then you can use a "Recall" inside the "Process Documents":

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="recall" compatibility="7.4.000" expanded="true" height="68" name="Recall" width="90" x="313" y="238">
<parameter key="name" value="wordnet"/>
<parameter key="io_object" value="WordnetDictionary"/>
<parameter key="remove_from_store" value="false"/>
</operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="wordnet:find_hypernym_wordnet" compatibility="5.3.000" expanded="true" height="82" name="Find Hypernyms (WordNet)" width="90" x="514" y="34">
<parameter key="use_prefix" value="true"/>
<parameter key="synset_word_prefix" value="hyper:"/>
<parameter key="maximum_recursion_depth" value="1"/>
<parameter key="multiple_meanings_per_word_policy" value="Take only first meaning"/>
<parameter key="multiple_synsets_policy" value="Take all synsets per meaning"/>
<parameter key="multiple_synset_words_policy" value="Take only first synset word"/>
<parameter key="concatenation" value="Concatenate result per synset"/>
<parameter key="keep_original_tokens" value="true"/>
<parameter key="keep_unmatched_tokens" value="false"/>
<parameter key="take_ID_instead_of_words" value="false"/>
<parameter key="work_on_type_noun" value="true"/>
<parameter key="work_on_type_verb" value="true"/>
<parameter key="work_on_type_adjective" value="true"/>
<parameter key="work_on_type_adverb" value="true"/>
</operator>
</process>