Silhouette Score for clustering Explained | Silhouette (clustering)- Validating Clustering Models

Silhouette Score for clustering Explained | Silhouette (clustering)- Validating Clustering Models
#SilhouetteScore #UnfoldDataScience
Hello ,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I speak about Silhouette Score and explain step by step how Silhouette Score works for cluster validation. I explain how to validate clusters and how to measure goodness of clusters. I explain the mathematical formula of Silhouette Score and intuition behind it. Below points are discussed in this video:
1. Silhouette Score for clustering
2. Validation on K-means clusters
3. Cluster validation techniques
4. How to measure goodness of clusters
5. Unsupervised machine learning accuracy
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
If you need Data Science training from scratch . Please fill this form (Please Note: Training is chargeable)
docs.google.com/forms/d/1Acua...
Book recommendation for Data Science:
Category 1 - Must Read For Every Data Scientist:
The Elements of Statistical Learning by Trevor Hastie - amzn.to/37wMo9H
Python Data Science Handbook - amzn.to/31UCScm
Business Statistics By Ken Black - amzn.to/2LObAA5
Hands-On Machine Learning with Scikit Learn, Keras, and TensorFlow by Aurelien Geron - amzn.to/3gV8sO9
Ctaegory 2 - Overall Data Science:
The Art of Data Science By Roger D. Peng - amzn.to/2KD75aD
Predictive Analytics By By Eric Siegel - amzn.to/3nsQftV
Data Science for Business By Foster Provost - amzn.to/3ajN8QZ
Category 3 - Statistics and Mathematics:
Naked Statistics By Charles Wheelan - amzn.to/3gXLdmp
Practical Statistics for Data Scientist By Peter Bruce - amzn.to/37wL9Y5
Category 4 - Machine Learning:
Introduction to machine learning by Andreas C Muller - amzn.to/3oZ3X7T
The Hundred Page Machine Learning Book by Andriy Burkov - amzn.to/3pdqCxJ
Category 5 - Programming:
The Pragmatic Programmer by David Thomas - amzn.to/2WqWXVj
Clean Code by Robert C. Martin - amzn.to/3oYOdlt
My Studio Setup:
My Camera : amzn.to/3mwXI9I
My Mic : amzn.to/34phfD0
My Tripod : amzn.to/3r4HeJA
My Ring Light : amzn.to/3gZz00F
Join Facebook group :
groups/41022...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/profile/Aman-Ku...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.com/drive/folder...
Have a different question for me? Ask me here : docs.google.com/forms/d/1ccgl...
My Music: www.bensound.com/royalty-free...

[@shaneglean217]

The explanation is so concise and simple, even in spite of the fact that english isn't your 1st language. Amazing work my dude.

[@chrisrichardson8864]

Clear and right to the point! Thank you for your work! Much appreciated!

[@UnfoldDataScience]

Thanks for watching Chris. Stay safe. Tc

[@nicolaevasiliu9430]

Amazing explanations! THANK YOU !!!
You are an excellent teacher !!!

[@Pannafreestyle]

Sir you are the best teacher for DS ! Keep up the great work

[@isaacyekoniya8067]

This is simple and well explained, thank you Aman!

[@UnfoldDataScience]

Most welcome!

[@JuliaCh47]

Thank you so much, best explanation I've seen on the Internet!

[@UnfoldDataScience]

Thanks a lot for watching.

[@KHakimin]

This is gold! thanks for putting it very simple to understand

[@UnfoldDataScience]

Glad it was helpful Khairul.

[@YouTribalChief]

What a gem of an explanation you give bhai! Truly you make things so simple and understandable!
You deserve more!

[@UnfoldDataScience]

Thanks a lot. Pls share with friends as well. Have a nice weekend ahead.

[@the.quantum.paradox]

Very simple and easy to understand!

[@Felipe90820]

Amazing explanation! Congrats and thank you!

[@dbapple]

thank you, straight to the point!

[@aaryalad7775]

I'm glad i found a teacher like you

[@samiayasmin5782]

Thank you so much. Simple and clean explanation.

[@victorcardenas9621]

Simple but powerful, thanks and greetings from Colombia!

[@Anon11229]

Too good...Way better than online courses that take thousands and thousands of money
keep up the hard work

[@UnfoldDataScience]

Thanks Zaid.

[@RoopaNandu-k8b]

Aman, your explanation is really good. Very precise and in detail

[@sanjithaamarathunga9949]

Thank you! Simple and well explained!

[@UnfoldDataScience]

Glad it was helpful!

[@Krishna-pm8ty]

Excellent Amen. Perfect explanation👏👏👏

[@mmmommm237]

Great, ty!

[@uchennanwosu5327]

Excellent illustration and explanation.

[@optimalsolutionprogramming1299]

Thank you for your help. Really great explanation!

[@UnfoldDataScience]

Welcome

[@ravisavaliya4333]

at first it was very difficult for me to understand this, but you explained this in a very easy way....THANKYOU AMAN SIR FOR YOUR EASY AND INTUITIVE EXPLANATION!!!

[@UnfoldDataScience]

It's my pleasure

[@kushik.naveen]

This is so useful for understanding & revising. Amazing work! Thankyou!

[@muhammedthayyib9202]

Clearly explained!

[@sangeethaagoudar2175]

Thank You so much for this video sir

[@keinestudiere-chillzone633]

Straight to the point. Thank you

[@UnfoldDataScience]

Thanks a lot for watching

[@krishnabhadke6161]

very well explained sir thank you

[@mdyounusahamed6668]

very concise and simple explanation.

[@RezoanurRahman]

You explained it so well even a 5th grader would get it. Its so easily explained!.

[@Suno-99]

beautiful explanation, keep it up

[@MyerNore]

Clearer than my professor, for sure

[@omniscienceisdead8837]

you are the best bro!

[@vallimuthaiyah5098]

Thank you sir for very clear explanation with example.. 👍

[@UnfoldDataScience]

Welcome Valli :)

[@akashpb4044]

Awesome ...plain and simple 👍🏼

[@UnfoldDataScience]

Glad to hear that Akash 🙂, please share with others as well who could be benefited from such content.

[@chrisx742]

Thank you.

[@lhadz7290]

Thank you so much for Sir, I am so glad I came across your video. I look forward to learn more from you. Thank you again.

[@UnfoldDataScience]

So nice of you

[@liqs_xd]

Great video, thanks for your time!

[@UnfoldDataScience]

My pleasure!

[@anshuraghav5620]

Great video

[@veyselaytekin8734]

thank you

[@parvsharma8767]

Thanks bro for explaining so perfectly

[@UnfoldDataScience]

Welcome Parv.

[@sourabhpramanik3957]

Its really a simple and good explanation for anybody to understand...thanks for making this simple explanation video.

[@UnfoldDataScience]

Welcome Sourabh

[@blackyogurt]

Great Explanation . thanks

[@amoghbharadwaj9252]

beautifully explained!!! Thank u so much

[@UnfoldDataScience]

Thanks Amogh. Happy Holi.

[@diyap7343]

This is amazing!

[@UnfoldDataScience]

Thanks Diya.

[@sajjaddehghani3280]

Thanks for the very simple and easy to understand explanation

[@UnfoldDataScience]

Welcome

[@user-hs6gz4yn1u]

Ur Explanation is simple and understandable,TQ sir

[@nandankakadiya1494]

Amazing explaination thank you sir

[@UnfoldDataScience]

Welcome Nandan.

[@shaikhuzma786]

Tqsm sir so well explained ❤

[@legolife3012]

Thanks a lot for this simple explenation

[@UnfoldDataScience]

You are welcome

[@user-dd3mf2sp7s]

Great explanation, Thank you!

[@UnfoldDataScience]

Welcome

[@almonddonut1818]

Thank you!!

[@UnfoldDataScience]

Welcome.

[@stevemungai3542]

Crystal clear

[@paulholzapfel1856]

Perfect explanation. Thank you Aman

[@UnfoldDataScience]

Welcome Paul. Your feedback is precious to me.

[@kavyasharma4738]

how can i thank you brother . you explained this topic so easily.

[@mohe4ever514]

Very well explained!

[@UnfoldDataScience]

Glad it was helpful!

[@theone-jz2rf]

keep up the good work!!!

[@UnfoldDataScience]

Thank you.

[@puodzius]

Very good explanation, thanks

[@UnfoldDataScience]

Welcome.

[@sandipansarkar9211]

finished watching

[@venkataprashanth9831]

great and simple explanation, a big clap to ur efforts

[@UnfoldDataScience]

Thanks Venkat.

[@anisadillap.7131]

oke i understand, thankyou sir ...

[@UnfoldDataScience]

Welcome

[@nandinik7816]

Always to the point...and you dont waste any time actually...

[@UnfoldDataScience]

Thanks again.

[@megalaramu]

Hi Aman, I do have a general question in clustering -I have heard of people do clustering in the dataset initially and then train separate model for each cluster. my question is lets take an example of telecom churn - there will be many services like internet, Phone, Dish etc. is there anyway we can cluster like internet users in one cluster, phone service in one cluster like that.

[@SACHINKUMAR-px8kq]

Thanks you so much sir

[@UnfoldDataScience]

Welcome Sachin.

[@Chaseaction-outdoor-activities]

thank you!!!

[@UnfoldDataScience]

Welcome.

[@dr.zliiiiii]

nice video

[@PranavVashisth]

verry well explained !!!!

[@UnfoldDataScience]

Thanks Pranav.

[@kamal_douma]

very very good explnation

[@user-ur2en1zq4f]

thanks sir

[@UnfoldDataScience]

Most welcome

[@dosendaring]

This is very good explanation, thank you Sir....

[@UnfoldDataScience]

Welcome Yudy.

[@beprepare8128]

Very well explained sir

[@UnfoldDataScience]

Thanks and welcome

[@MohdRashid-hi7cp]

Very nice explanation

[@UnfoldDataScience]

Thanks Rashid.pls share with friends also

[@jeanhwang18]

Thank you so much for your explaination. I have a small question: after I got point i ' s silhouette coefficient, I calculate other two points in cluster A, then I take the average of the three silhouette coefficients, let's say it's NO.1. What we can learn from No.1? If No.1 is close to +1, it's good, it means data points in cluster A are most similar to each other, right?

[@bangarrajumuppidu8354]

excellent

[@UnfoldDataScience]

Thanks a lot.

[@salilshukla08]

thanks

[@UnfoldDataScience]

Welcome.

[@shaheerrehman3151]

Thank you. Much love to you from Pakistan

[@UnfoldDataScience]

Thanks Shaheer, pls share with friends if you like the content.

[@prashanths4455]

super bro

[@UnfoldDataScience]

Thanks Prashanth

[@vishwass5103]

Hi Aman thanks for this explanation. Please explain about assumptions of k means clustering

[@UnfoldDataScience]

Assumption at high level is, you data has clusters and centriods.

[@iftikhar58]

clear

[@UnfoldDataScience]

Thank you

[@gnaneshgn8341]

Nice video Sir thank you. please make an video of EDA sir ..!

[@UnfoldDataScience]

Thanks Ganesh.

[@handsomemehdi3445]

Thank u for simpler explanation! A question: In our case, it seems that for both clusters, b should be the same unless we have more than two clusters, (or since those clusters are mutually nearest) doesn't it?

[@lkjhlkjh1954]

I think that’s why we do not calculate silhouettes score for less than 2 cluster

[@bipulnath4602]

Amazing explaination..plz make a video on Dunn index.

[@UnfoldDataScience]

Thanks Bipul.

[@neajinnovations8587]

I got all the concept clear, but please when is it appropriate to use silhouette distance? it when you have mixed data? or mix data is best for elbow method?

[@ranajaydas8906]

Sir please make a video on PCA . It would be very helpful

[@UnfoldDataScience]

Sure Ranajay.

[@Live_life_with_no_regrets]

is there any way to run it in SPSS? Is it correct if I use the average of the Euclidean distance of the cluster?

[@christopherlegarda5164]

This guy is god!

[@sampathvinaykumarreddymajj790]

Topic is very well explained sir ..
But should we learn DB Index, Dunn Index, Jaccard score ... or only this Silhouette score is enough

[@UnfoldDataScience]

Thanks Sampath.

[@TheGentlemanGR]

So which score is better? -0.2 or 0.21 ?

[@subhasisdutta3711]

you have said "Min distance of A1,A2,A3 is the value of b ". Here you are talking about mean value of three or the which one has the minimum value among three , we'll chose it. Please clear it once, if possible

[@UnfoldDataScience]

HI Subhasis, A1, A2 etc are "mean" Distance from various other clusters ok. Now let's take att these A1 TO AN values and take the minimum from this set. This mimum is b.

[@subhasisdutta3711]

@@UnfoldDataScience thanq for the clarification. Have a blessed life ahead

[@mahipatil369]

hello sir.. great video i understand this topic very well,, thank you so much...
just want to ask did you make video of implementation of silhoutte score in python?? if yes plz provide link..

[@UnfoldDataScience]

Thanks a lot. Its very simple to do in python - see this link - stackoverflow.com/questions/59919627/how-to-calculate-the-silhouette-score-for-each-cluster-separately-in-python

[@parikshitgurjar5545]

hello sir, Plese make a video on the explanation and implementation of ppf,pdf,cdf in python with simple explanation and understanding, I am very confused in this.

[@UnfoldDataScience]

pdf cdf video is available Parikshit. Please search in my channel.

[@parikshitgurjar5545]

@@UnfoldDataScience Thankyou sir❤️

[@himanshumangoli6708]

As we have find cohesion and separation for only one point in one cluster A, so we have to find Silhouette score for all points for all points in same cluster so just to ensure that points are correctly clustered ??
Please explain??

[@UnfoldDataScience]

Yes , one point is just for example.

[@pruthvi491]

How to get silhouette score for spectral clustering

[@UnfoldDataScience]

I Will check.

[@arisurya9867]

Nice video sir. But i have problem when calculate cohesion in cluster just has one data point. How i should set value of cohesion? 0 or 1?

[@UnfoldDataScience]

Thats not a good clustering process, clusters need to be reformed.

[@arisurya9867]

Thank you sir 🙏

[@gretchenbekker2371]

How do I get the amount of data points within the clusters ?

[@UnfoldDataScience]

Levels can be found out after clustering then we can count using pandas.

[@nerdymath6]

is 0.4 is good silhouette score?

[@UnfoldDataScience]

Dont think so

[@abeerhamid]

How can i read the graph of silhouette score? Most important thing you didn't tell

[@UnfoldDataScience]

Positive Scores (close to 1): Samples are well-clustered, and there is good separation between clusters.
Scores around 0: Samples are on or near the boundary between clusters, indicating potential overlap.
Negative Scores: Samples are likely in the wrong cluster.

[@dariomelconian9502]

Is 0.39 a bad score?

[@UnfoldDataScience]

can be better

[@sameeransari9129]

apki english samajhne me time lg jata hai.. ki aap kya bolna chahte ho

[@sameeransari9129]

Sir aap hindi me samjhaya karo toh zyada aacha samjha payoge...

[@sameeransari9129]

apki english me confuse ho jate hai log ki aap kya bolna chahte ho...isko negitive comment ki tarah mt lena...aap samjhate aacha ho pr agar hindi me samjhaoge toh hm logo ko clear samaj ayega