two-way analysis of variance

In 1925, Ronald Fisher mentions the two-way ANOVA in his celebrated book, Statistical Methods for Research Workers (chapters 7 and 8). In 1934, Frank Yates published procedures for the unbalanced case.{{cite journal |last=Yates |first=Frank |date=March 1934 |title=The analysis of multiple classifications with unequal numbers in the different classes |jstor=2278459 |journal=Journal of the American Statistical Association |volume=29 |issue=185 |pages=51–66 |doi=10.1080/01621459.1934.10502686}} Since then, an extensive literature has been produced. The topic was reviewed in 1993 by Yasunori Fujikoshi.{{cite journal |last=Fujikoshi |first=Yasunori |date=1993 |title=Two-way ANOVA models with unbalanced data |journal=Discrete Mathematics |volume=116 |issue=1 |pages=315–334 |doi=10.1016/0012-365X(93)90410-U |doi-access=free }} In 2005, Andrew Gelman proposed a different approach of ANOVA, viewed as a multilevel model.{{cite journal |last=Gelman |first=Andrew |date=February 2005 |title=Analysis of variance? why it is more important than ever |journal=The Annals of Statistics |volume=33 |issue=1 |pages=1–53 | arxiv=math/0504499|doi=10.1214/009053604000001048 |s2cid=125025956 }}

Let us imagine a data set for which a dependent variable may be influenced by two factors which are potential sources of variation. The first factor has $I$ levels {{nowrap|($i\; \backslash in\; \backslash \{1,\backslash ldots,I\backslash \}$)}} and the second has $J$ levels {{nowrap|($j\; \backslash in\; \backslash \{1,\backslash ldots,J\backslash \}$)}}. Each combination $(i,j)$ defines a treatment, for a total of $I\; \backslash times\; J$ treatments. We represent the number of replicates for treatment $(i,j)$ by $n\_\{ij\}$, and let $k$ be the index of the replicate in this treatment {{nowrap|($k\; \backslash in\; \backslash \{1,\backslash ldots,n\_\{ij\}\backslash \}$)}}.

From these data, we can build a contingency table, where $n\_\{i+\}\; =\; \backslash sum\_\{j=1\}^J\; n\_\{ij\}$ and $n\_\{+j\}\; =\; \backslash sum\_\{i=1\}^I\; n\_\{ij\}$, and the total number of replicates is equal to $n\; =\; \backslash sum\_\{i,j\}\; n\_\{ij\}\; =\; \backslash sum\_i\; n\_\{i+\}\; =\; \backslash sum\_j\; n\_\{+j\}$.

The experimental design is balanced if each treatment has the same number of replicates, $K$. In such a case, the design is also said to be orthogonal, allowing to fully distinguish the effects of both factors. We hence can write $\backslash forall\; i,j\; \backslash ;\; n\_\{ij\}\; =\; K$, and $\backslash forall\; i,j\; \backslash ;\; n\_\{ij\}\; =\; \backslash frac\{n\_\{i+\}\; \backslash cdot\; n\_\{+j\}\}\{n\}$.

Upon observing variation among all $n$ data points, for instance via a histogram, "probability may be used to describe such variation".{{cite journal |last=Kass |first=Robert E |date=1 February 2011 |title=Statistical inference: The big picture |journal=Statistical Science |volume=26 |issue=1 |pages=1–9 |doi=10.1214/10-sts337|pmid=21841892 |pmc=3153074 |arxiv=1106.2895 }} Let us hence denote by $Y\_\{ijk\}$ the random variable which observed value $y\_\{ijk\}$ is the $k$-th measure for treatment $(i,j)$. The two-way ANOVA models all these variables as varying independently and normally around a mean, $\backslash mu\_\{ij\}$, with a constant variance, $\backslash sigma^2$ (homoscedasticity):

$Y\_\{ijk\}\; \backslash ,\; |\; \backslash ,\; \backslash mu\_\{ij\},\; \backslash sigma^2\; \backslash ;\; \backslash overset\{\backslash mathrm\{i.i.d.\}\}\{\backslash sim\}\; \backslash ;\; \backslash mathcal\{N\}(\backslash mu\_\{ij\},\; \backslash sigma^2)$.

Specifically, the mean of the response variable is modeled as a linear combination of the explanatory variables:

$\backslash mu\_\{ij\}\; =\; \backslash mu\; +\; \backslash alpha\_i\; +\; \backslash beta\_j\; +\; \backslash gamma\_\{ij\}$,

where $\backslash mu$ is the grand mean, $\backslash alpha\_i$ is the additive main effect of level $i$ from the first factor (i-th row in the contingency table), $\backslash beta\_j$ is the additive main effect of level $j$ from the second factor (j-th column in the contingency table) and $\backslash gamma\_\{ij\}$ is the non-additive interaction effect of treatment $(i,j)$ for samples $k=1,...,n\_\{ij\}$ from both factors (cell at row i and column j in the contingency table).

Another equivalent way of describing the two-way ANOVA is by mentioning that, besides the variation explained by the factors, there remains some statistical noise. This amount of unexplained variation is handled via the introduction of one random variable per data point, $\backslash epsilon\_\{ijk\}$, called error. These $n$ random variables are seen as deviations from the means, and are assumed to be independent and normally distributed:

$Y\_\{ijk\}\; =\; \backslash mu\_\{ij\}\; +\; \backslash epsilon\_\{ijk\}\; \backslash text\{\; with\; \}\; \backslash epsilon\_\{ijk\}\; \backslash overset\{\backslash mathrm\{i.i.d.\}\}\{\backslash sim\}\; \backslash mathcal\{N\}(0,\; \backslash sigma^2)$.

Following Gelman and Hill, the assumptions of the ANOVA, and more generally the general linear model, are, in decreasing order of importance:{{cite book |last1=Gelman |first1=Andrew |last2=Hill |first2=Jennifer|author2-link=Jennifer Hill |date=18 December 2006 |title= Data Analysis Using Regression and Multilevel/Hierarchical Models |url=http://www.cambridge.org/us/academic/subjects/statistics-probability/statistical-theory-and-methods/data-analysis-using-regression-and-multilevelhierarchical-models |publisher=Cambridge University Press |pages=45–46 |isbn=978-0521867061 }}

1. the data points are relevant with respect to the scientific question under investigation;
2. the mean of the response variable is influenced additively (if not interaction term) and linearly by the factors;
3. the errors are independent;
4. the errors have the same variance;
5. the errors are normally distributed.

To ensure identifiability of parameters, we can add the following "sum-to-zero" constraints:

$\backslash sum\_i\; \backslash alpha\_i\; =\; \backslash sum\_j\; \backslash beta\_j\; =\; \backslash sum\_i\; \backslash gamma\_\{ij\}\; =\backslash sum\_j\; \backslash gamma\_\{ij\}=\; 0$

In the classical approach, testing null hypotheses (that the factors have no effect) is achieved via their significance which requires calculating sums of squares.

Testing if the interaction term is significant can be difficult because of the potentially-large number of degrees of freedom.{{cite journal |author=Yi-An Ko|date=September 2013 |title=Novel Likelihood Ratio Tests for Screening Gene-Gene and Gene-Environment Interactions with Unbalanced Repeated-Measures Data |journal=Genetic Epidemiology |volume=37 |issue=6 |pages=581–591 |doi=10.1002/gepi.21744 |pmid=23798480 |display-authors=etal|pmc=4009698}}

The following hypothetical example gives the yields of 15 plants subject to two different environmental variations, and three different fertilisers.

Five sums of squares are calculated:

Finally, the sums of squared deviations required for the analysis of variance can be calculated.{{cite book|last=Mecklin|first=Christopher|title=STA 265 Notes (Methods of Statistics and Data Science)|date=20 October 2020|access-date=6 December 2024|chapter-url=https://bookdown.org/cmecklin/sta265notes/anova-with-interaction.html|chapter=Chapter 7: ANOVA with Interaction|via=bookdown.org}}

• Analysis of variance
• F-test (Includes a one-way ANOVA example)
• Mixed model
• Multivariate analysis of variance (MANOVA)
• One-way ANOVA
• Repeated measures ANOVA
• Tukey's test of additivity

{{Reflist}}

• {{cite book |author=George Casella |date=18 April 2008 |title=Statistical design |url=https://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-75964-7 |publisher=Springer |isbn=978-0-387-75965-4 |series=Springer Texts in Statistics |author-link=George Casella }}

Category:Analysis of variance