Sliced inverse regression

Given a response variable $\backslash ,Y$ and a (random) vector $X\; \backslash in\; \backslash R^p$ of explanatory variables, SIR is based on the model$$Y=f(\backslash beta\_1^\backslash top\; X,\backslash ldots,\backslash beta\_k^\backslash top\; X,\backslash varepsilon)\backslash quad\backslash quad\backslash quad\backslash quad\backslash quad(1)$$where $\backslash beta\_1,\backslash ldots,\backslash beta\_k$ are unknown projection vectors, $\backslash ,k$ is an unknown number smaller than $\backslash ,p$, $\backslash ;f$ is an unknown function on $\backslash R^\{k+1\}$as it only depends on$\backslash ,k$ arguments, and $\backslash varepsilon$ is a random variable representing error with $E[\backslash varepsilon|X]=0$ and a finite variance of $\backslash sigma^2$. The model describes an ideal solution, where $\backslash ,Y$ depends on $X\; \backslash in\; \backslash R^p$ only through a$\backslash ,k$ dimensional subspace; i.e., one can reduce the dimension of the explanatory variables from$\backslash ,p$ to a smaller number$\backslash ,k$ without losing any information.

An equivalent version of $\backslash ,(1)$ is: the conditional distribution of $\backslash ,Y$ given $\backslash ,\; X$ depends on $\backslash ,\; X$ only through the $\backslash ,k$ dimensional random vector $(\backslash beta\_1^\backslash top\; X,\backslash ldots,\backslash beta\_k^\backslash top\; X)$. It is assumed that this reduced vector is as informative as the original $\backslash ,X$ in explaining $\backslash ,\; Y$.

The unknown $\backslash ,\backslash beta\_i\text{'}s$ are called the effective dimension reducing directions (EDR-directions). The space that is spanned by these vectors is denoted by the effective dimension reducing space (EDR-space).

Given $\backslash underline\{a\}\_1,\backslash ldots,\backslash underline\{a\}\_r\; \backslash in\; \backslash R^n$, then $V:=L(\backslash underline\{a\}\_1,\backslash ldots,\backslash underline\{a\}\_r)$, the set of all linear combinations of these vectors is called a linear subspace and is therefore a vector space. The equation says that vectors $\backslash underline\{a\}\_1,\backslash ldots,\backslash underline\{a\}\_r$ span $\backslash ,V$, but the vectors that span space $\backslash ,V$ are not unique.

The dimension of $\backslash ,V\; (\backslash in\; \backslash R^n)$ is equal to the maximum number of linearly independent vectors in $\backslash ,V$. A set of $\backslash ,n$ linear independent vectors of $\backslash R^n$ makes up a basis of $\backslash R^n$. The dimension of a vector space is unique, but the basis itself is not. Several bases can span the same space. Dependent vectors can still span a space, but the linear combinations of the latter are only suitable to a set of vectors lying on a straight line.

Computing the inverse regression curve (IR) means instead of looking for

• $\backslash ,E[Y|X=x]$, which is a curve in $\backslash R^p$

it is actually

• $\backslash ,E[X|Y=y]$, which is also a curve in $\backslash R^p$, but consisting of $\backslash ,p$ one-dimensional regressions.

The center of the inverse regression curve is located at $\backslash ,E[E[X|Y]]=E[X]$. Therefore, the centered inverse regression curve is

• $\backslash ,E[X|Y=y]-E[X]$

which is a $\backslash ,p$ dimensional curve in $\backslash R^p$.

The centered inverse regression curve lies on a $\backslash ,k$-dimensional subspace spanned by $\backslash ,\backslash Sigma\_\{xx\}\backslash beta\_i\backslash ,\text{'}s$. This is a connection between the model and inverse regression.

Given this condition and $\backslash ,(1)$, the centered inverse regression curve $\backslash ,E[X|Y=y]-E[X]$ is contained in the linear subspace spanned by $\backslash ,\backslash Sigma\_\{xx\}\backslash beta\_k(k=1,\backslash ldots,K)$, where $\backslash ,\backslash Sigma\_\{xx\}=Cov(X)$.

After having had a look at all the theoretical properties, the aim now is to estimate the EDR-directions. For that purpose, weighted principal component analyses are needed. If the sample means $\backslash ,\backslash hat\{m\}\_h\backslash ,\text{'}s$, $\backslash ,X$ would have been standardized to $\backslash ,Z=\backslash Sigma\_\{xx\}^\{-1/2\}\backslash \{X-E(X)\backslash \}$. Corresponding to the theorem above, the IR-curve $\backslash ,m\_1(y)=E[Z|Y=y]$ lies in the space spanned by $\backslash ,(\backslash eta\_1,\backslash ldots,\backslash eta\_k)$, where $\backslash ,\backslash eta\_i=\backslash Sigma^\{1/2\}\_\{xx\}\; \backslash beta\_i$. As a consequence, the covariance matrix $\backslash ,cov[E[Z|Y]]$ is degenerate in any direction orthogonal to the $\backslash ,\backslash eta\_i\backslash ,\text{'}s$. Therefore, the eigenvectors $\backslash ,\backslash eta\_k\; (k=1,\backslash ldots,K)$ associated with the largest$\backslash ,K$ eigenvalues are the standardized EDR-directions.

The algorithm from Li, K-C. (1991) to estimate the EDR-directions via SIR is as follows.

1. Let $\backslash ,\backslash Sigma\_\{xx\}$ be the covariance matrix of $\backslash ,X$. Standardize $\backslash ,X$ to

: $\backslash ,Z=\backslash Sigma\_\{xx\}^\{-1/2\}\backslash \{X-E(X)\backslash \}$

$\backslash ,(1)$ can also be rewritten as

: $Y=f(\backslash eta\_1^\backslash top\; Z,\backslash ldots,\backslash eta\_k^\backslash top\; Z,\backslash varepsilon)$

where $\backslash ,\backslash eta\_k=\backslash beta\_k\backslash Sigma\_\{xx\}^\{1/2\}\backslash quad\backslash forall\backslash ;\; k$.)

2. Divide the range of $\backslash ,y\_i$ into $\backslash ,S$ non-overlapping slices $\backslash ,H\_s(s=1,\backslash ldots,S).\backslash ;\; n\_s$ is the number of observations within each slice and $\backslash ,I\_\{H\_s\}$ is the indicator function for the slice:

: $n\_s=\backslash sum\_\{i=1\}^n\; I\_\{H\_s\}(y\_i)$

3. Compute the mean of $\backslash ,z\_i$ over all slices, which is a crude estimate $\backslash ,\backslash hat\{m\}\_1$ of the inverse regression curve $\backslash ,m\_1$:

: $\backslash ,\backslash bar\{z\}\_s=n\_s^\{-1\}\backslash sum\_\{i=1\}^n\; z\_i\; I\_\{H\_s\}(y\_i)$

4. Calculate the estimate for $\backslash ,Cov\backslash \{m\_1(y)\backslash \}$:

: $\backslash ,\backslash hat\{V\}=n^\{-1\}\backslash sum\_\{i=1\}^S\; n\_s\; \backslash bar\{z\}\_s\; \backslash bar\{z\}\_s^\backslash top$

5. Identify the eigenvalues $\backslash ,\backslash hat\{\backslash lambda\}\_i$ and the eigenvectors $\backslash ,\backslash hat\{\backslash eta\}\_i$ of $\backslash ,\backslash hat\{V\}$, which are the standardized EDR-directions.

6. Transform the standardized EDR-directions back to the original scale. The estimates for the EDR-directions are given by:

: $\backslash ,\backslash hat\{\backslash beta\}\_i=\backslash hat\{\backslash Sigma\}\_\{xx\}^\{-1/2\}\backslash hat\{\backslash eta\}\_i$

(which are not necessarily orthogonal.)

{{Reflist}}

• Li, K-C. (1991) "Sliced Inverse Regression for Dimension Reduction", Journal of the American Statistical Association, 86, 316–327 [https://www.jstor.org/stable/2290563 Jstor]
• Cook, R.D. and Sanford Weisberg, S. (1991) "Sliced Inverse Regression for Dimension Reduction: Comment", Journal of the American Statistical Association, 86, 328–332 [https://www.jstor.org/stable/2290564 Jstor]
• Härdle, W. and Simar, L. (2003) Applied Multivariate Statistical Analysis, Springer Verlag. {{ISBN|3-540-03079-4}}
• Kurzfassung zur Vorlesung Mathematik II im Sommersemester 2005, A. Brandt

Category:Regression analysis

Category:Dimension reduction