The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Operator nominal to numerical error

BifruanBifruan Member Posts: 2 Learner I
in Help
My data is 700,000 records with 8 Attributes.
When I start to operate "nominal to numerical" the error is occur that "need more memory"
but my computer is 8 GB with 512 SSD and Core I5 Gen 11
Is that my spec of my computer is not enough for this operation?
what I can do to fix this problem?
Tagged:

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    Hi,
    how many different classes are in your column? If you have a 700.000 different values in one column, than this operator simply creates 700.000 new columns. This means you roughly increase your data set by the factor 700.000.

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • BifruanBifruan Member Posts: 2 Learner I
    There are 5000 different classes in one column so is it possible that rapidminer can create 5000 new column.
    And Is that student license affect to limit of the operation rows?

    Best Regards
  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    edited February 2021
    no it does come from your edu license.You increased your data set size from
    700.000x8 = 5.6 Million data entries
    to
    700.000x5000 = 3.5 Billion data entries.
    Thats just killing your laptop. I would either choose something like target encoding or try to reduce the number of different classes i.e. with Replace Rare Values or manual mapping

    Best,
    Martin


    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • MarcoBarradasMarcoBarradas Administrator, Employee-RapidMiner, RapidMiner Certified Analyst, Member Posts: 272 Unicorn
    @Bifruan hi what type of model are you trying to build, from the picture you uploaded i see that you are taking a basket ID in the operator. I'm guessing you are using some kind of transactions dataset, may basket analysis, why do you need to convert your ID to individual columns?


Sign In or Register to comment.