The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Process Speed up

Muhammed_Fatih_Muhammed_Fatih_ Member Posts: 93 Maven
in Help
Hello together, 

is there a possibility to speed up a process? I am currently conducting a dimensionality reduction process with SVD - the process took 5 days and is still running. Is that normal? 

 Thank you in advance for your help! 
Tagged:

Answers

  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    hi @Muhammed_Fatih_ so the answer to this is "it depends". How many rows and columns are in your data set? What are the capabilities of your computer? Sometimes RapidMiner needs hours to go through large data sets - it depends.

    Scott

  • Muhammed_Fatih_Muhammed_Fatih_ Member Posts: 93 Maven
    The matrix (word vector) has 9662 attribute columns (terms) and in total 72826 rows. My SVD process is running for four days and has not finished yet. Especially the SVD operator itself is still loading. I have a 32 GB memory on my computer and an i7 core.
  • sgenzersgenzer Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    @Muhammed_Fatih_ yeah so that's a LOT of columns. If it's a result of text mining (word vectors), why don't you use pruning to reduce dimensionality rather than SVD?
  • Muhammed_Fatih_Muhammed_Fatih_ Member Posts: 93 Maven
    My research goal is to evaluate the effect of different dimensionaity reduction algorithms.

    But beside that - very interesting hint: Which criteria would you use for pruning? Where is the cut to set?
Sign In or Register to comment.