The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here

Robustness metrics

teioteio Member Posts: 2 Learner I
in Help

Hello,

Is there any metric provided to assess robustness / stability of a ML module? As far as I understand the ones provided are more related with the functionality.
I understand reliability as the ability to perform the intended function in the presence of abnormal or unknown inputs, related to ‘reliability’.

Any comment is welcome
Thanks in advance

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee-RapidMiner, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,533 RM Data Scientist
    Hi there,
    anything in particular you search for? You could for example use SHAPly and see how much the output changes?

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • teioteio Member Posts: 2 Learner I
    Hello,

    Thanks for your answer. I have to analyse it but it could work. I am Quality Assurance people and we are searching for metrics provided by the commercial tools to assess the quality of a ML/AI model. In our research, the metrics provided by the mostly used tools in the market, are related to functionality and not to others characteristics mentioned in ISO/IEC standards for AI systems as Robustness, Transparency, User Controllability.

    Our first option is always to use a tool to asssess quality than manual.

    Thanks for your support, Regards Teresa
Sign In or Register to comment.