Silhouette (clustering)

File:Silhouette-plot-orange.png data mining suite. At the bottom of the plot, silhouette identifies dolphin and porpoise as outliers in the group of mammals.]]

Assume the data have been clustered via any technique, such as k-medoids or k-means, into $k$ clusters.

For data point $i\; \backslash in\; C\_I$ (data point $i$ in the cluster $C\_I$), let

: $a(i)\; =\; \backslash frac\{1\}$

to be the smallest (hence the $\backslash min$ operator in the formula) mean distance of $i$ to all points in any other cluster (i.e., in any cluster of which $i$ is not a member). The cluster with this smallest mean dissimilarity is said to be the "neighboring cluster" of $i$ because it is the next best fit cluster for point $i$.

We now define a silhouette (value) of one data point $i$

:$s(i)\; =\; \backslash frac\{b(i)\; -\; a(i)\}\{\backslash max\backslash \{a(i),b(i)\backslash \}\}$, if $|C\_I|\; >\; 1$

and

:$s(i)\; =\; 0$, if $|C\_I|\; =\; 1$

Which can be also written as:

:$s(i)\; =\; \backslash begin\{cases\}$

1-a(i)/b(i), & \mbox{if } a(i) < b(i) \\

0, & \mbox{if } a(i) = b(i) \\

b(i)/a(i)-1, & \mbox{if } a(i) > b(i) \\

\end{cases}

From the above definition it is clear that

:$-1\; \backslash le\; s(i)\; \backslash le\; 1$

Note that $a(i)$ is not clearly defined for clusters with size = 1, in which case we set $s(i)=0$. This choice is arbitrary, but neutral in the sense that it is at the midpoint of the bounds, -1 and 1.

For $s(i)$ to be close to 1 we require $a(i)\; \backslash ll\; b(i)$. As $a(i)$ is a measure of how dissimilar $i$ is to its own cluster, a small value means it is well matched. Furthermore, a large $b(i)$ implies that $i$ is badly matched to its neighbouring cluster. Thus an $s(i)$ close to 1 means that the data is appropriately clustered. If $s(i)$ is close to -1, then by the same logic we see that $i$ would be more appropriate if it was clustered in its neighbouring cluster. An $s(i)$ near zero means that the datum is on the border of two natural clusters.

The mean $s(i)$ over all points of a cluster is a measure of how tightly grouped all the points in the cluster are. Thus the mean $s(i)$ over all data of the entire dataset is a measure of how appropriately the data have been clustered. If there are too many or too few clusters, as may occur when a poor choice of $k$ is used in the clustering algorithm (e.g., k-means), some of the clusters will typically display much narrower silhouettes than the rest. Thus silhouette plots and means may be used to determine the natural number of clusters within a dataset. One can also increase the likelihood of the silhouette being maximized at the correct number of clusters by re-scaling the data using feature weights that are cluster specific.{{Cite journal

| doi = 10.1016/j.ins.2015.06.039

| author = R.C. de Amorim, C. Hennig

| title = Recovering the number of clusters in data sets with noise features using feature rescaling factors

| journal = Information Sciences

| volume = 324

| pages = 126–145

| year = 2015| arxiv = 1602.06989

| s2cid = 315803

}}

Kaufman et al. introduced the term silhouette coefficient for the maximum value of the mean $s(i)$ over all data of the entire dataset,{{cite book

|author1=Leonard Kaufman

|author2=Peter J. Rousseeuw

|author2-link=Peter J. Rousseeuw

|title=Finding groups in data : An introduction to cluster analysis

|date=1990

|publisher=Wiley-Interscience

|location=Hoboken, NJ

|isbn=9780471878766

|doi=10.1002/9780470316801

|page=[https://archive.org/details/findinggroupsind00kauf/page/87 87]

|url=https://archive.org/details/findinggroupsind00kauf/page/87

}}

i.e.,

:$$

SC = \max_k \tilde{s}\left(k\right),

where $\backslash tilde\{s\}\backslash left(k\backslash right)$ represents the mean $s(i)$ over all data of the entire dataset for a specific number of clusters $k$.

Computing the silhouette coefficient needs all $\backslash mathcal\{O\}(N^2)$ pairwise distances, making this evaluation much more costly than clustering with k-means. For a clustering with centers $\backslash mu\_\{C\_I\}$ for each cluster $C\_I$, we can use the following simplified Silhouette for each point $i\backslash in\; C\_I$ instead, which can be computed using only $\backslash mathcal\{O\}(Nk)$ distances:

:$a\text{'}(i)=d(i,\backslash mu\_\{C\_I\})$ and $b\text{'}(i)=\backslash min\_\{C\_J\backslash neq\; C\_I\}\; d(i,\backslash mu\_\{C\_J\})$,

which has the additional benefit that $a\text{'}(i)$ is always defined, then define accordingly the simplified silhouette and simplified silhouette coefficient{{Cite conference |last1=Hruschka |first1=E.R. |last2=de Castro |first2=L.N. |last3=Campello |first3=R.J.G.B. |date=2004 |title=Evolutionary Algorithms for Clustering Gene-Expression Data |url=https://ieeexplore.ieee.org/document/1410321 |conference=Fourth IEEE International Conference on Data Mining (ICDM'04) |publisher=IEEE |pages=403–406 |doi=10.1109/ICDM.2004.10073|url-access=subscription }}

:$s\text{'}(i)\; =\; \backslash frac\{b\text{'}(i)\; -\; a\text{'}(i)\}\{\backslash max\backslash \{a\text{'}(i),b\text{'}(i)\backslash \}\}$

:$SC\text{'}\; =\; \backslash max\_k\; \backslash frac1N\; \backslash sum\_i\; s\text{'}\backslash left(i\backslash right)$.

If the cluster centers are medoids (as in k-medoids clustering) instead of arithmetic means (as in k-means clustering), this is also called the medoid-based silhouette{{Cite journal |last1=Van der Laan |first1=Mark |last2=Pollard |first2=Katherine |last3=Bryan |first3=Jennifer |date=2003 |title=A new partitioning around medoids algorithm |url=http://www.tandfonline.com/doi/abs/10.1080/0094965031000136012 |journal=Journal of Statistical Computation and Simulation |language=en |volume=73 |issue=8 |pages=575–584 |doi=10.1080/0094965031000136012 |s2cid=17437463 |issn=0094-9655|url-access=subscription }} or medoid silhouette.{{Cite conference |last1=Lenssen |first1=Lars |last2=Schubert |first2=Erich |date=2022 |title=Clustering by Direct Optimization of the Medoid Silhouette |url=https://link.springer.com/10.1007/978-3-031-17849-8_15 |conference=International Conference on Similarity Search and Applications |language=en |pages=190–204 |doi=10.1007/978-3-031-17849-8_15 |access-date=2022-10-20|arxiv=2209.12553 }}

If every object is assigned to the nearest medoid (as in k-medoids clustering), we know that $a\text{'}(i)\backslash leq\; b\text{'}(i)$, and hence $s\text{'}(i)\; =\; \backslash frac\{b\text{'}(i)\; -\; a\text{'}(i)\}\{b\text{'}(i)\}\; =\; 1\; -\; \backslash frac\{a\text{'}(i)\}\{b\text{'}(i)\}$.

Instead of using the average silhouette to evaluate a clustering obtained from, e.g., k-medoids or k-means, we can try to directly find a solution that maximizes the Silhouette. We do not have a closed form solution to maximize this, but it will usually be best to assign points to the nearest cluster as done by these methods. Van der Laan et al. proposed to adapt the standard algorithm for k-medoids, PAM, for this purpose and call this algorithm PAMSIL:

1. Choose initial medoids by using PAM
2. Compute the average silhouette of this initial solution
3. For each pair of a medoid {{Math|m}} and a non-medoid {{Math|x}}
4. swap {{Math|m}} and {{Math|x}}
5. compute the average silhouette of the resulting solution
6. remember the best swap
7. un-swap {{Math|m}} and {{Math|x}} for the next iteration
8. Perform the best swap and return to 3, otherwise stop if no improvement was found.

The loop in step 3 is executed for $\backslash mathcal\{O\}(Nk)$ pairs, and involves computing the silhouette in $\backslash mathcal\{O\}(N^2)$, hence this algorithm needs $\backslash mathcal\{O\}(N^3ki)$ time, where {{Math|i}} is the number of iterations.

Because this is a fairly expensive operation, the authors propose to also use the medoid-based silhouette, and call the resulting algorithm PAMMEDSIL. It needs $\backslash mathcal\{O\}(N^2k^2i)$ time.

Batool et al. propose a similar algorithm under the name OSil, and propose a CLARA-like sampling strategy for larger data sets, that solves the problem only for a sub-sample.{{Cite journal |last1=Batool |first1=Fatima |last2=Hennig |first2=Christian |date=2021 |title=Clustering with the Average Silhouette Width |url=https://linkinghub.elsevier.com/retrieve/pii/S0167947321000244 |journal=Computational Statistics & Data Analysis |language=en |volume=158 |pages=107190 |doi=10.1016/j.csda.2021.107190|arxiv=1910.11339 |s2cid=219260336 }}

By adopting recent improvements to the PAM algorithm, FastMSC reduces the runtime using the medoid silhouette to just $\backslash mathcal\{O\}(N^2i)$.

By starting with a maximum number of clusters {{Math|k{{sub|max}}}} and iteratively removing the worst center (in terms of a change in silhouette) and re-optimizing, the best (highest medoid silhouette) clustering can be automatically determined. As data structures can be reused, this reduces the computation cost substantially over repeatedly running the algorithm for different numbers of clusters.{{Cite journal |last1=Lenssen |first1=Lars |last2=Schubert |first2=Erich |date=2024-02-01 |title=Medoid Silhouette clustering with automatic cluster number selection |url=https://www.sciencedirect.com/science/article/pii/S0306437923001266 |journal=Information Systems |volume=120 |pages=102290 |doi=10.1016/j.is.2023.102290 |issn=0306-4379|arxiv=2309.03751 }}

This algorithm needs pairwise distances and is typically implemented with a pairwise distance matrix. The $\backslash mathcal\{O\}(N^2)$ memory requirement is the main limiting factor for applying this to very large data sets.

• Cluster analysis
• Davies–Bouldin index
• Calinski-Harabasz index
• Dunn index
• Determining the number of clusters in a data set

{{Reflist}}

{{Machine learning evaluation metrics}}

Category:Clustering criteria