Uncertainty coefficient
{{distinguish|Theil index}}
In statistics, the uncertainty coefficient, also called proficiency, entropy coefficient or Theil's U, is a measure of nominal association. It was first introduced by Henri Theil{{citation needed|date=July 2012}} and is based on the concept of information entropy.
Definition
Suppose we have samples of two discrete random variables, X and Y. By constructing the joint distribution, {{math|PX,Y(x, y)}}, from which we can calculate the conditional distributions, {{math|1=PX{{!}}Y(x{{!}}y) = PX,Y(x, y)/PY(y)}} and {{math|1=PY{{!}}X(y{{!}}x) = PX,Y(x, y)/PX(x)}}, and calculating the various entropies, we can determine the degree of association between the two variables.
The entropy of a single distribution is given as:
{{cite book
|author1=Claude E. Shannon
|author2=Warren Weaver
|year=1963
|title=The Mathematical Theory of Communication
|publisher=University of Illinois Press
}}
:
H(X)= -\sum_x P_X(x) \log P_X(x) ,
while the conditional entropy is given as:
:
H(X|Y) = -\sum_{x,~y} P_{X,Y}(x,~y) \log P_{X|Y}(x|y) .
{{cite book
|title=Numerical Recipes: the Art of Scientific Computing
|author1=William H. Press
|author2=Brian P. Flannery
|author3=Saul A. Teukolsky
|author4=William T. Vetterling
|year=1992
|publisher=Cambridge University Press
|edition=3rd
|page=761
|section=14.7.4
|url=http://apps.nrbook.com/empanel/index.html#pg=761
}} or proficiency{{Cite journal
| last1= White |first1= Jim | last2= Steingold | first2=Sam | last3= Fournelle | first3=Connie
| title = Performance Metrics for Group-Detection Algorithms
| series = Interface 2004
| url = http://www.interfacesymposia.org/I04/I2004Proceedings/WhiteJim/WhiteJim.paper.pdf
| archive-url = https://web.archive.org/web/20120413090855/http://www.interfacesymposia.org/I04/I2004Proceedings/WhiteJim/WhiteJim.paper.pdf
| url-status = usurped
| archive-date = April 13, 2012
}} is defined as:
:
U(X|Y) = \frac{H(X)-H(X|Y)}{H(X)} = \frac{I(X;Y)}{H(X)} ,
and tells us: given Y, what fraction of the bits of X can we predict? In this case we can think of X as containing the total information, and of Y as allowing one to predict part of such information.
The above expression makes clear that the uncertainty coefficient is a normalised mutual information I(X;Y). In particular, the uncertainty coefficient ranges in [0, 1] as I(X;Y) < H(X) and both I(X,Y) and H(X) are positive or null.
Note that the value of U (but not H!) is independent of the base of the log since all logarithms are proportional.
The uncertainty coefficient is useful for measuring the validity of a statistical classification algorithm and has the advantage over simpler accuracy measures such as precision and recall in that it is not affected by the relative fractions of the different classes, i.e., P(x).
{{Cite journal
|last1 = Peter
|first1 = Mills
|title = Efficient statistical classification of satellite measurements
|journal = International Journal of Remote Sensing
|doi = 10.1080/01431161.2010.507795
|year = 2011
|volume = 32
|issue = 21
|pages = 6109–6132
|url = http://peteysoft.users.sourceforge.net/TRES_A_507795.pdf
|arxiv = 1202.2194
|bibcode = 2011IJRS...32.6109M
|s2cid = 88518570
|url-status = dead
|archiveurl = https://web.archive.org/web/20120426073755/http://peteysoft.users.sourceforge.net/TRES_A_507795.pdf
|archivedate = 2012-04-26
}}
It also has the unique property that it won't penalize an algorithm for predicting the wrong classes, so long as it does so consistently (i.e., it simply rearranges the classes). This is useful in evaluating clustering algorithms since cluster labels typically have no particular ordering.
Variations
The uncertainty coefficient is not symmetric with respect to the roles of X and Y. The roles can be reversed and a symmetrical measure thus defined as a weighted average between the two:
:
\begin{align}
U(X,~Y) & = \frac{H(X)U(X|Y)+H(Y)U(Y|X)}{H(X)+H(Y)} \\[8pt]
& = 2 \left [\frac{H(X) + H(Y) - H(X,~Y)}{H(X)+H(Y)} \right ] .
\end{align}
Although normally applied to discrete variables, the uncertainty coefficient can be extended to continuous variables using density estimation.{{citation needed|date=July 2012}}
See also
References
External links
- [http://libagf.sourceforge.net libagf] Includes software for calculating uncertainty coefficients.