User:Coopeajj

The Frish-Waugh-Lovell Theorem states that if the regression we are concerned with is:

:$\backslash textstyle\; \backslash mathbf\{y\}\; =\; \backslash mathbf\{X\_1\}\backslash mathbf\{\backslash beta\_1\}\; +\; \backslash mathbf\{X\_2\}\backslash mathbf\{\backslash beta\_2\}\; +\; \backslash mathbf\{u\}\; \backslash !$

Where $\backslash scriptstyle\; \backslash mathbf\{X\_1\}$ and $\backslash scriptstyle\; \backslash mathbf\{X\_2\}$ are $\backslash scriptstyle\; n\; \backslash times\; k\_1$ and $\backslash scriptstyle\; n\; \backslash times\; k\_2$ respectively and where $\backslash scriptstyle\; \backslash mathbf\{\backslash beta\_1\}$ and $\backslash scriptstyle\; \backslash mathbf\{\backslash beta\_2\}$ are conformable.

Then, the estimate of $\backslash scriptstyle\; \backslash mathbf\{\backslash beta\_2\}$ will be the same as the estimate from a modified regression of the form:

:$\backslash textstyle\; \backslash mathbf\{M\_1\}\backslash mathbf\{y\}\; =\; \backslash mathbf\{M\_1\}\backslash mathbf\{X\_2\}\backslash mathbf\{\backslash beta\_2\}\; +\; \backslash mathbf\{M\_1\}\backslash mathbf\{u\}\; \backslash !$

Where, $\backslash scriptstyle\; \backslash mathbf\{M\_1\}$ projects onto the orthogonal compliment of $\backslash scriptstyle\; \backslash mathbf\{X\_1\}$. Specifically,

$\backslash textstyle\; \backslash mathbf\{M\_1\}=\backslash mathbf\{\{X\_1\}(\{\{\{X\_1\}^T\}\{X\_1\}\})^\{-1\}\{X\_1\}^T\}$

This result is extremely useful as it implies that all these secondary regressions are not necessary (i.e., using projection matrices to make the variables orthogonal to each other will lead one to the exact same results as just running the regression with all non-orthogonal included).

If we are only concerned with $\backslash scriptstyle\; \backslash beta\_2$ we must make matrices $\backslash scriptstyle\; \backslash mathbf\{X\_1\}$ and $\backslash scriptstyle\; \backslash mathbf\{X\_2\}$ orthogonal. I.e., we can use orthogonal projection matricies to anihilate $\backslash scriptstyle\; \backslash mathbf\{X\_1\}$.

Define:

:$\backslash textstyle\; \backslash mathbf\{P\_X\}\; =\; \backslash mathbf\{X(X^TX)^\{-1\}X^T\}$

:$\backslash textstyle\; \backslash mathbf\{\{I\_n\}\; -\; \{P\_X\}=\{M\_X\}\}$

Clearly, $\backslash scriptstyle\; \backslash mathbf\{\{M\_X\}X=0\}$ (i.e.,

$\backslash scriptstyle\; \backslash mathbf\{M\_X\}$ projects onto the orthogonal compliment of $\backslash scriptstyle\; \backslash mathbf\{X\}$.)

To estimate the parameters using OLS they