The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Filtering ALL Examples (no. of missing data in the attribute = total no. of examples)

1705410G1705410G Member Posts: 43 Learner III
edited January 2019 in Help
I would like to filter out all missing data. Where the number of missing data in the attribute is equal to the total number of examples. 


I know Select Attribute can do the trick. However, as I have roughly 20-30 attributes where all examples missing, so I would not like to pick it out individually. 


I've tried using Filter Examples > no_missing_attributes, but I have no idea why I ended up with no examples at all from 2041 examples.

Thank you!

regards,
AY
Tagged:

Best Answer

Answers

  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi @1705410G,

    Could you share your dataset and eventually your process ?

    Regards,

    Lionel
  • 1705410G1705410G Member Posts: 43 Learner III
    edited January 2019
    NIL - Sorry, had to remove due to privacy. 
  • 1705410G1705410G Member Posts: 43 Learner III
    edited January 2019
    lionelderkrikor 

    I'm not sure if this is the correct way to share a process. I achieve 0 examples after running this process when I only want to remove those attributes with ALL missing examples.


    Regards,
    AY
  • lionelderkrikorlionelderkrikor RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn
    Hi again @1705410G,

    A solution is to use Remove Useless Attributes operator...

    Hope it helps,

    Regards,

    Lionel
  • 1705410G1705410G Member Posts: 43 Learner III
    @ lionelderkrikor 

    Thank you!!

    Regards,
    AY
  • tftemmetftemme Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 164 RM Research
    Hi @1705410G,

    Another solution is the operator Filter Attributes with Missing Values from the Operator Toolbox extension (which exactly does what you want and has further configurations to remove Attributes with Missing Values). The reason that you end up with 0 Examples when using Filter Examples is that the operator removes examples for which the attribute selected is missing. So as your attribute you actually want to remove has only missing values, all Examples are removed.

    Best regards,
    Fabian

  • 1705410G1705410G Member Posts: 43 Learner III
    @ lionelderkrikor ,

    What parameters should I set in Remove Useless Attributes then? Or should I leave it ? I read it briefly but i'm unsure how it works.

    Regards,
    AY

  • 1705410G1705410G Member Posts: 43 Learner III
    @ lionelderkrikor ,

    What parameters should I set in Remove Useless Attributes then? Or should I leave it ? I read it briefly but I don't understand how it works.

    Regards, 
    AY

  • 1705410G1705410G Member Posts: 43 Learner III
    @ lionelderkrikor ,

    What parameters should I set it too? I read it up briefly but am unsure how it works.

    Regards,
    AY
  • 1705410G1705410G Member Posts: 43 Learner III
    @ lionelderkrikor ,

    What parameters should I set it too? I read it up briefly but am unsure how it works.

    Regards,
    AY
  • Telcontar120Telcontar120 RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    You can leave the default settings and it should eliminate any constant attributes.  Try running that and see if it gives you the results you are expecting.  If you then want to set a lower threshold then you can modify the min deviation for numericals or nominals.
    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
Sign In or Register to comment.