The Altair Community is migrating to a new platform to provide a better experience for you. In preparation for the migration, the Altair Community is on read-only mode from October 28 - November 6, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here
Ellbow Plot
Muhammed_Fatih_
Member Posts: 93 
Maven
Dear community,
I want to determine or rather apporximate the optimal k for k-means and tried to generate an Ellbow Plot. The generated plot looks like the following which confused me a bit:
It is not so easy to detect an optimal k due to the reason that more than one ellbows were generated and the graph doesn't look like a conventional Ellbow Plot. Where to set the k? At k=6, k=7 or k=14 where a brakdown is respectively marked with regard to the avg.centroid distance measure. Or would it be better to generate an Ellbow Plot by considering Davies Bouldin insetead of Centroid Distance?
The data is represented by TF-IDF values.
Thanks in advance for your hints!
Best regards!
I want to determine or rather apporximate the optimal k for k-means and tried to generate an Ellbow Plot. The generated plot looks like the following which confused me a bit:
It is not so easy to detect an optimal k due to the reason that more than one ellbows were generated and the graph doesn't look like a conventional Ellbow Plot. Where to set the k? At k=6, k=7 or k=14 where a brakdown is respectively marked with regard to the avg.centroid distance measure. Or would it be better to generate an Ellbow Plot by considering Davies Bouldin insetead of Centroid Distance?
The data is represented by TF-IDF values.
Thanks in advance for your hints!
Best regards!
Tagged:
0
Answers
Intuitively I would say :
k = 3 (the first slope failure)
OR
like you k = 7
Regards,
Lionel
thank you for recommendation!
Would you assess the given course of the graph as "normal"? Is it generally possible to have more than one ellows? What are your experiences with regard to this?
Best regards!
In deed, your curve is not usual.
In most of the cases, the Elbow curve looks like the curves presented in this thread :
https://community.rapidminer.com/discussion/comment/61654#Comment_61654
Regards,
Lionel
Thank you for the link. I know this thread. My Ellbow curve obviously differs from the presented one in the thread. But what might be potential reasons for getting such a curve?
Best regards!