Dixon's Q test

{{Short description|Criterion for identification and rejection of outliers}}

{{more citations needed|date=May 2015}}

In statistics, Dixon's Q test, or simply the Q test, is used for identification and rejection of outliers. This assumes normal distribution and per Robert Dean and Wilfrid Dixon, and others, this test should be used sparingly and never more than once in a data set. To apply a Q test for bad data, arrange the data in order of increasing values and calculate Q as defined:

: Q = \frac{\text{gap}}{\text{range}}

Where gap is the absolute difference between the outlier in question and the closest number to it. If Q > Qtable, where Qtable is a reference value corresponding to the sample size and confidence level, then reject the questionable point. Note that only one point may be rejected from a data set using a Q test.

Example

Consider the data set:

:0.189,\ 0.167,\ 0.187,\ 0.183,\ 0.186,\ 0.182,\ 0.181,\ 0.184,\ 0.181,\ 0.177 \,

Now rearrange in increasing order:

:0.167,\ 0.177,\ 0.181,\ 0.181,\ 0.182,\ 0.183,\ 0.184,\ 0.186,\ 0.187,\ 0.189 \,

We hypothesize that 0.167 is an outlier. Calculate Q:

:Q=\frac{\text{gap}}{\text{range}} = \frac

0.177-0.167
{0.189-0.167}=0.455.

With 10 observations and at 90% confidence, Q = 0.455 > 0.412 = Qtable, so we conclude 0.167 is indeed an outlier. However, at 95% confidence, Q = 0.455 < 0.466 = Qtable 0.167 is not considered an outlier.

McBaneHalpern, Arthur M. "Experimental physical chemistry : a laboratory textbook." 3rd ed. / Arthur M. Halpern, George C. McBane. New York : W. H. Freeman, c2006 [http://lccn.loc.gov/2006286239 Library of Congress]{{dead link|date=December 2016 |bot=InternetArchiveBot |fix-attempted=yes }} notes: Dixon provided related tests intended to search for more than one outlier, but they are much less frequently used than the r10 or Q version that is intended to eliminate a single outlier.

Table

This table summarizes the limit values of the two-tailed Dixon's Q test.

style="vertical-align: top;" | Number of values:

| style="vertical-align: top; text-align: center;" |  3

| style="vertical-align: top; text-align: center;" | 4

| style="vertical-align: top; text-align: center;" | 5

| style="vertical-align: top; text-align: center;" | 6

| style="vertical-align: top; text-align: center;" | 7

| style="vertical-align: top; text-align: center;" | 8

| style="vertical-align: top; text-align: center;" | 9

| style="vertical-align: top; text-align: center;" | 10

----

| style="vertical-align: top;" | Q90%:

| style="vertical-align: top; text-align: center;" | 0.941

| style="vertical-align: top; text-align: center;" | 0.765

| style="vertical-align: top; text-align: center;" | 0.642

| style="vertical-align: top; text-align: center;" | 0.560

| style="vertical-align: top; text-align: center;" | 0.507

| style="vertical-align: top; text-align: center;" | 0.468

| style="vertical-align: top; text-align: center;" | 0.437

| style="vertical-align: top; text-align: center;" | 0.412

----

| style="vertical-align: top;" | Q95%:

| style="vertical-align: top; text-align: center;" | 0.970

| style="vertical-align: top; text-align: center;" | 0.829

| style="vertical-align: top; text-align: center;" | 0.710

| style="vertical-align: top; text-align: center;" | 0.625

| style="vertical-align: top; text-align: center;" | 0.568

| style="vertical-align: top; text-align: center;" | 0.526

| style="vertical-align: top; text-align: center;" | 0.493

| style="vertical-align: top; text-align: center;" | 0.466

----

| style="vertical-align: top;" | Q99%:

| style="vertical-align: top; text-align: center;" | 0.994

| style="vertical-align: top; text-align: center;" | 0.926

| style="vertical-align: top; text-align: center;" | 0.821

| style="vertical-align: top; text-align: center;" | 0.740

| style="vertical-align: top; text-align: center;" | 0.680

| style="vertical-align: top; text-align: center;" | 0.634

| style="vertical-align: top; text-align: center;" | 0.598

| style="vertical-align: top; text-align: center;" | 0.568

See also

References

{{Reflist}}

Further reading

  • Robert B. Dean and Wilfrid J. Dixon (1951) "Simplified Statistics for Small Numbers of Observations". Anal. Chem., 1951, 23 (4), 636–638. [http://pubs.acs.org/doi/abs/10.1021/ac60052a025 Abstract] [http://depa.fquim.unam.mx/amyd/archivero/ac1951_23_636_13353.pdf Full text PDF] {{Webarchive|url=https://web.archive.org/web/20150501042023/http://depa.fquim.unam.mx/amyd/archivero/ac1951_23_636_13353.pdf |date=2015-05-01 }}
  • Rorabacher, D. B. (1991) "Statistical Treatment for Rejection of Deviant Values: Critical Values of Dixon Q Parameter and Related Subrange Ratios at the 95 percent Confidence Level". Anal. Chem., 63 (2), 139–146. [http://pubs.acs.org/doi/pdf/10.1021/ac00002a010 PDF] (including larger tables of limit values)
  • McBane, George C. (2006) "Programs to Compute Distribution Functions and Critical Values for Extreme Value Ratios for Outlier Detection". J. Statistical Software 16(3):1–9, 2006 [http://www.jstatsoft.org/v16/i03 Article (PDF) and Software (Fortan-90, Zipfile)]
  • Shivanshu Shrivastava, A. Rajesh, P. K. Bora (2014) "Sliding window Dixon's tests for malicious users' suppression in a cooperative spectrum sensing system" IET Communications, 2014, 8 (7)
  • W. J. Dixon. The Annals of Mathematical Statistics. Vol. 21, No. 4 (Dec., 1950), pp. 488–506 {{doi| 10.1214/aoms/1177729747}}