User:Michael Hardy/Envelope model

The envelope model was originally introduced under the framework of multivariate linear regression:

:$Y=\backslash alpha+\backslash beta\; X+\backslash varepsilon,$

where $Y\backslash in\; \backslash mathbb\{R\}^\{r\}$ is the random response vector and $X\backslash in\; \backslash mathbb\{R\}^p$ is the non-stochastic predictor vector centered to have mean zero. Under this model, $\backslash alpha\backslash in\; \backslash mathbb\{R\}^\{r\}$ and $\backslash beta\backslash in\; \backslash mathbb\{R\}^\{r\backslash times\; p\}$ are the unknown intercept and coefficient parameter respectively. The error vector $\backslash varepsilon\backslash in\; \backslash mathbb\{R\}^\{r\}$ is assumed to have mean $\backslash mathbf\{0\}$ and covariance $\backslash Sigma\backslash in\; \backslash mathbb\{R\}^\{r\backslash times\; r\}$, where $\backslash Sigma$ is a positive-definite matrix.

Let $\backslash mathcal\{S\}\backslash subseteq\; \backslash mathbb\{R\}^\{r\}$ be a subspace that divides $Y$ into two disjoint parts: the material part $P\_\{\backslash mathcal\{S\}\}Y$ and the immaterial part $Q\_\{\backslash mathcal\{S\}\}Y$. $P\_\{\backslash mathcal\{S\}\}$ is the projection matrix onto $\backslash mathcal\{S\}$ and $Q\_\{\backslash mathcal\{S\}\}=I\_r-P\_\{\backslash mathcal\{S\}\}$, where $I\_r$ is the $r\backslash times\; r$ identity matrix, and $Q\_\{\backslash mathcal\{S\}\}$ is the projection onto the orthogonal complement of $\backslash mathcal\{S\}$ (denoted by $\backslash mathcal\{S\}^\{\backslash bot\}$). We assume that the following two conditions hold:

Q_{\mathcal{S}}Y\mid X \sim Q_{\mathcal{S}}Y,

Q_{\mathcal{S}} Y ~\bot~ P_{\mathcal{S}}Y\mid X,

where "A ~ B" means A and B are identically distributed, "A | B" denotes the conditional distribution of A given B and $A\; \backslash mathbin\{\backslash bot\}\; B$ denotes that A and B are statistically independent.

Assumption 1 states that the conditional distribution of the immaterial part given $X$ is the same as the distribution of the immaterial part. Assumption 2 states that given $X$, the material part and immaterial part are independent of each other.

Under the linear model, the first assumption indicates that the immaterial part does not give any information about $\backslash beta$, while the second assumption means that the covariance matrix of $Y\backslash mid\; X$ (i.e. the covariance matrix of the error) can be decomposed into two parts: the variability according to the material part and the variability according to immaterial part. Since $Y$ can be decomposed as $Y\; =\; Q\_\{\backslash mathcal\{S\}\}Y+P\_\{\backslash mathcal\{S\}\}Y$, we see that $P\_\{\backslash mathcal\{S\}\}Y$ carries all the material information and maybe some immaterial information about $\backslash beta$, while $Q\_\{\backslash mathcal\{S\}\}Y$ carries just immaterial information about $\backslash beta$. To ensure that we exclude immaterial information from our estimate of $\backslash beta$, we aim to the find the smallest $\backslash mathcal\{S\}$ that carries only material information about $\backslash beta$.

In 2010 Cook et al.{{cite journal | title=Envelope Models for parsimonious and Efficient Multivariate Linear Regression| last=Cook | first=R.D. |author2=Li, B. |author3=Chiaromonte,F.| journal=Statistica Sinica | volume=20 | year=2010 | pages=927–960 | citeseerx=10.1.1.362.5659| issue=3 | jstor=2240725}}

showed that assumptions 1 and 2 are equivalent to

{{ordered list|type=lower-roman

|$\backslash mathcal\{B\}\; \backslash subseteq\; \backslash mathcal\{S\},$ where $\backslash mathcal\{B\}\; =\; \backslash operatorname\{span\}(\backslash beta),$

|$\backslash Sigma=\backslash Sigma\_\{\backslash mathcal\{S\}\}+\backslash Sigma\_\{\backslash mathcal\{S\}^\backslash bot\}=P\_S\backslash Sigma\; P\_\{\backslash mathcal\{S\}\}\; +\; Q\_\{\backslash mathcal\{S\}\}\; \backslash Sigma\; Q\_\{\backslash mathcal\{S\}\}.$

}}

In (i), the span denotes the linear subpace spanned by the columns of $\backslash beta$. When (ii) holds, $\backslash mathcal\{S\}$ is called a reducing subspace of $\backslash Sigma$.

The $\backslash Sigma$-envelope of $\backslash mathcal\{B\}$, denoted by $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})$, is defined as the intersection of all reducing subspaces of $\backslash Sigma$ that contain $\backslash mathcal\{B\}$.

We let $u$ denote the dimension of envelope subspace, where $u\; \backslash leq\; r$. From (ii), it is clear that $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})$ decomposes $\backslash Sigma$ into the variation of the material and immaterial parts of $Y$. Namely, we have $\backslash Sigma\_\{\backslash mathcal\{S\}\}=\backslash operatorname\{var\}(P\_\{\backslash mathcal\{S\}\}Y\backslash mid\; X)$ and $\backslash Sigma\_\{\backslash mathcal\{S\}^\backslash bot\}=\backslash operatorname\{var\}(Q\_\{\backslash mathcal\{S\}\}Y\backslash mid\; X)$.

Let $\backslash mathbb\{S\}^\{r\backslash times\; r\}$ denote the class of all symmetric $r\backslash times\; r$ matrices. The following propositions establish some important properties of the envelope subspace $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{S\})$, namely its relationship with the eigenstructure of $\backslash Sigma$ and its behavior under linear transformations of $\backslash mathcal\{S\}$.

• Proposition: Let $\backslash Sigma\backslash in\; \backslash mathbb\{S\}^\{r\backslash times\; r\}$ and let $P\_i,\; i=1,\backslash ldots,q,$ be the projection onto the eigenspaces of $\backslash Sigma$, and let $\backslash mathcal\{S\}$ be a subspace of $\backslash operatorname\{span\}(\backslash Sigma)$. Then $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{S\})=\backslash sum\_\{i=1\}^\{q\}P\_i\; \backslash mathcal\{S\}$ is the intersection of all reducing subspaces of $\backslash Sigma$ that contain $\backslash mathcal\{S\}$.

• Proposition: Let $K\backslash in\; \backslash mathbb\{S\}^\{r\backslash times\; r\}$ be the matrix that commute with $\backslash Sigma\backslash in\; \backslash mathbb\{S\}^\{r\backslash times\; r\}$ and let $\backslash mathcal\{S\}$ be a subspace of span$(\backslash Sigma)$. Then $K\backslash mathcal\{S\}\backslash subseteq\; \backslash operatorname\{span\}(\backslash Sigma)$ and the following equivariance holds: $\backslash mathcal\{E\}\_\backslash Sigma(K\backslash mathcal\{S\})=K\backslash mathcal\{E\}\_\backslash Sigma\; (\backslash mathcal\{S\})$. If, in addition, $K\backslash mathcal\{S\}\backslash subseteq\; \backslash operatorname\{span\}(K)$ and $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{S\})$ reduces $K$, then the following invariance holds: $\backslash mathcal\{E\}\_\{\backslash Sigma\}(K\backslash mathcal\{S\})=\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{S\})$.

• Proposition: Under the standard model, let $\backslash Sigma\_\{Y\}$ be the covariance matrix of $Y$. Then $\backslash Sigma^\{-1\}\backslash mathcal\{B\}=\backslash Sigma\_\{Y\}^\{-1\}\backslash mathcal\{B\},$ and: $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})=\backslash mathcal\{E\}\_\{\backslash Sigma\_Y\}(\backslash mathcal\{B\})=\backslash mathcal\{E\}\_\backslash Sigma(\backslash Sigma^\{-1\}\backslash mathcal\{B\})=\backslash mathcal\{E\}\_\{\backslash Sigma\_Y\}(\backslash Sigma\_Y^\{-1\}\backslash mathcal\{B\})=\backslash mathcal\{E\}\_\{\backslash Sigma\_Y\}(\backslash Sigma^\{-1\}\backslash mathcal\{B\})=\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash Sigma\_Y^\{-1\}\backslash mathcal\{B\}).$

Under the envelope parameterization, the multivariate linear regression model (which we call the standard model) can be written as

: $$

\begin{align}

Y & =\alpha+\Gamma\eta X+\varepsilon, \\[6pt]

\Sigma & =\Gamma\Omega\Gamma^T+\Gamma_0\Omega_0 \Gamma^T_0,

\end{align}

where $\backslash beta=\backslash Gamma\backslash eta$, $\backslash Gamma\backslash in\; \backslash mathbb\{R\}^\{r\backslash times\; u\}$ is an orthonormal basis for the envelope subspace $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})$, and $\backslash Gamma\_\{0\}\backslash in\; \backslash mathbb\{R\}^\{r\backslash times\; (r-u)\}$ is an orthonormal basis for the orthogonal complement of the envelope subspace $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})^\{\backslash bot\}$ and the completion of $\backslash Gamma$ (that is, $(\backslash Gamma,\backslash Gamma\_\{0\})\backslash in\backslash mathbb\{R\}^\{r\backslash times\; r\}$ is an orthogonal matrix).

$\backslash eta\; \backslash in\; \backslash mathbb\{R\}^\{u\backslash times\; p\}$ carries the coordinates of $\backslash beta$ with respect to $\backslash Gamma$ and $\backslash Omega\backslash in\; \backslash mathbb\{R\}^\{u\backslash times\; u\}$ and $\backslash Omega\_\{0\}\backslash in\; \backslash mathbb\{R\}^\{(r-u)\backslash times\; (r-u)\}$ are positive definite matrices which carry the coordinates of $\backslash Sigma$ with respect to $\backslash Gamma$ and $\backslash Gamma\_\{0\}$ respectively.

Under the standard model, the total number of parameters is

:$N(r)=r+rp+\backslash frac\{r(r+1)\}\{2\},$

where $r$ is the number of parameters in $\backslash alpha$, $rp$ is the number of parameters in $\backslash beta$, and $\backslash frac\{r(r+1)\}\{2\}$ is the number of parameters in $\backslash Sigma$ which is the covariance of $\backslash varepsilon$.

Under the envelope model, the total number of parameters is

:$N(u)=r+up+\backslash frac\{r(r+1)\}\{2\}$

where $r$ is the number of parameters in $\backslash alpha$, $u(r-u)$ is the number of parameters in $\backslash operatorname\{span\}(\backslash Gamma)$, $up$ is the number of parameters in $\backslash eta$, $\backslash frac\{u(u+1)\}\{2\}$ is the number of parameters in $\backslash Omega$ and $\backslash frac\{(r-u)(r-u+1)\}\{2\}$ is the number of parameters in $\backslash Omega\_\{0\}$. When we add them together, we obtain $N(u)$.

Therefore, $N(r)-\; N(u)\; =\; p(r-u)$. So if , the envelope model reduces the number of parameters that need to be estimated. If $u=r$, the envelope model reduces to the standard model. If $u=0$, it means that $\backslash beta=0$.

The envelope model parameterization does not depend on normality. However, if we assume that the errors are normally distributed, i.e. $\backslash varepsilon\; \backslash sim\; N(\backslash mathbf\{0\},\backslash Sigma)$, then we can use maximum likelihood estimation (MLE) to estimate the unknown parameters: $\backslash alpha$, $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})$, $\backslash eta$,$\backslash Omega$ and $\backslash Omega\_\{0\}$.

In the envelope model, $\backslash Gamma$ is not identifiable. However, we can uniquely estimate $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})$. In order to estimate $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})$, we need to solve the following optimization problem:

:$\backslash hat\{\backslash mathcal\{E\}\}\_\backslash Sigma\; (\backslash mathcal\{B\})\; =\; \backslash displaystyle\; \backslash underset\{\{\backslash operatorname\{span\}\; (\backslash Gamma)\; \backslash in\; \backslash mathbb\{G\}^\{r\; \backslash times\; u\}\}\}\{\backslash operatorname\{argmin\}\}\; \backslash log\; |\; \backslash Gamma^T\; S\_\{Y\backslash mid\; X\}\; \backslash Gamma\; |\; +\; \backslash log\; |\; \backslash Gamma^T\; S\_\{Y\}^\{-1\}\; \backslash Gamma\; |,$

where $S\_\{Y\backslash mid\; X\}$ is the sample covariance matrix of the residuals from ordinary least squares regression and $S\_Y$ is the sample covariance of $Y$, $\backslash mathbb\{G\}^\{r\; \backslash times\; u\}$ denotes an $r\backslash times\; u$ Grassmann manifold $u$ in $\backslash mathbb\{R\}^r$ (that is, the collection of all $u$-dimensional subspaces of $\backslash mathbb\{R\}^r$).

If we do not have normality we can still use the above objective function to estimate the envelope model. If the error has finite fourth moments then we can still get a root-n consistent estimator of the parameters. This objective function can be solved by some standard manifold optimization algorithm such as [https://arxiv.org/pdf/1308.5200.pdf Manopt ]. Recently, a very fast algorithm for estimating $\backslash hat\{\backslash mathcal\{E\}\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})$, which does not require optimization over a Grassmannian, has also been developed.{{cite journal | title= A Note on Fast Envelope Estimation | last=Cook | first=R.D. |author2=Forzani, L.|author3=Su, Z.| journal= Journal of Multivariate Analysis | volume=150 | year=2016 | pages=42-54}}

Let $\backslash hat\{\backslash mathcal\{E\}\}\_\backslash Sigma(\backslash mathcal\{B\})$ denote the minimizer of this objective function and $\backslash hat\{\backslash Gamma\}$ be the orthonormal basis of $\backslash hat\{\backslash mathcal\{E\}\}\_\backslash Sigma(\backslash mathcal\{B\})$. Then $\backslash hat\{\backslash eta\}=\backslash hat\{\backslash Gamma\}^T\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$, where $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$ denotes the ordinary least squares (OLS) estimator $\backslash mathbb\{Y\}\_C^T\; \backslash mathbb\{X\}(\backslash mathbb\{X\}^\{T\}\backslash mathbb\{X\})^\{-1\}$, $\backslash mathbb\{Y\}\_C\; \backslash in\; \backslash mathbb\{R\}^\{n\; \backslash times\; r\}$ is the centered data matrix of the response (that is, every element in the $i^\backslash text\{th\}$ row of $\backslash mathbb\{Y\}\_C$ is the $i^\backslash text\{th\}$ data point in the sample of $\backslash mathbb\{Y\}$ minus the sample mean $\backslash bar\{\backslash mathbb\{Y\}\}$) and $\backslash mathbb\{X\}\backslash in\; \backslash mathbb\{R\}^\{n\; \backslash times\; p\}$ is the centered data matrix of the predictor. The envelope estimator of $\backslash beta$ is then $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}=\backslash hat\{\backslash Gamma\}\backslash hat\{\backslash eta\}$.

We see that the envelope estimator $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}$ is the projection of the OLS estimator (i.e. the estimator under the standard model) onto the estimated envelope subspace which removes the immaterial information. Since the immaterial information is identified and removed in subsequent analysis, the envelope estimator $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}$ is potentially more efficient than $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$. If $u\; =r$, then the envelope estimator reduces to the OLS estimator, i.e. $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}\; =\; \backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$. For the full derivation, refer to Cook et al. (2010).

In Cook et al. it is shown that under normality assumptions, the asymptotic variance of $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}$ satisfies

:$\backslash operatorname\{avar\}(\backslash operatorname\{vec\}(\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}))\backslash leq\; \backslash operatorname\{avar\}(\backslash operatorname\{vec\}(\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}))$

where $\backslash operatorname\{vec\}(A)$ is the vectorization of matrix $A$ and $\backslash operatorname\{avar\}(\backslash cdot)$ is the asymptotic variance; that is, if $\backslash sqrt\{n\}(T-\backslash theta)\; \backslash mathrel\{\backslash xrightarrow\{\backslash mathcal\{D\}\}\}\; N(0,A)$, then $\backslash operatorname\{avar\}(T)=A$. This means, the envelope estimator performs at least as good as the OLS estimator in terms of efficiency.

In particular, if $\backslash |\backslash Omega\backslash |\backslash ll\; \backslash |\backslash Omega\_0\backslash |$, where $\backslash |\backslash cdot\backslash |$ represents the matrix spectral norm, then the envelope estimator is expected to provide substantial efficiency gains. That is, if the immaterial part is more variant than material part, by identifying and removing the immaterial information, we expect to realize substantial efficieny gain (i.e. much smaller standard errors).

The dimension $u$ of $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})$ can be chosen by different methods such as likelihood ratio testing (LRT), Akaike information criterion (AIC), Bayesian information criterion (BIC), or cross-validation.

To illustrate the efficiency gains by envelope models, consider the problem of estimating two means of two bivariate normal populations, $Y\_1\; \backslash sim\; N(\backslash mu\_1,\backslash Sigma)$ and $Y\_2\; \backslash sim\; N(\backslash mu\_2,\backslash Sigma)$. This can be studied in the envelope framework by considering $\backslash mathbb\{Y\}=(\backslash mathbb\{Y\_1\},\backslash mathbb\{Y\_2\})^\{T\}$ as a bivariate response vector and $X$ as an indicator variable taking value $X=0$ in the first population and $X=1$ in the second population. We can parametrize in such a way that $\backslash alpha=\backslash mu\_1$ is the mean of the first population and $\backslash beta=\backslash mu\_2-\backslash mu\_1$ is the mean difference. Therefore, the multivariate linear model is

:$Y=\backslash mu\_1+(\backslash mu\_2-\backslash mu\_1)X+\backslash varepsilon=\backslash alpha+\backslash beta\; X+\backslash varepsilon.$

The standard estimator of $\backslash beta$ is the difference in the sample means for the two populations $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}=\backslash bar\{Y\}\_\{2\}-\backslash bar\{Y\}\_\{1\}$ and the corresponding estimator of $\backslash Sigma$ is the intra-sample covariance matrix. In the standard model, the estimator $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$ does not make use of the dependence between the responses and is equivalent to performing two univariate two univariate regressions of $Y\_j$ on $X,j=1,2$.

File:Envelope_Estimation2.jpg

In contrast, by considering envelope model, we obtain a different estimator for $\backslash beta$ that accounts for the dependence structure of $Y\_\{1\}$ and $Y\_\{2\}$ and that removes all the immaterial information to the group differences. The envelope estimator, $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}$ has smaller standard errors than $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}$, indicating the estimation for $\backslash beta$ is greatly improved. The graphical illustration of those analysis is shown in the figure. For further analysis, refer to

Cook, R.D. [http://users.stat.umn.edu/~rdcook/Stat8931F15/EnvCh%201.pdf An Introduction to envelope Models and Methods]

In the figure, without loss of generality, it is assumed $\backslash mu\_\{1\}+\backslash mu\_2=0$. Therefore, $\backslash mathcal\{B\}=\backslash operatorname\{span\}(\backslash mu\_1-\backslash mu\_2)=\backslash operatorname\{span\}(\backslash beta)$. The left panel illustrats the standard analysis. It directly projects the data $Y$ onto the $Y\_2$ axis, following the dashed line marked A, and then proceeds with inference based on the resulting univariate sample. The curves along the horizontal axis in the left panel stand for the projected distributions from the two populations. A standard analysis might involve constructing a two-sample t-test on samples drawn from these populations. There is considerable overlap between the two projected distributions, so it may take a large sample size to infer that $\backslash beta\_2\backslash neq\; 0$ (or $\backslash mu\_1\; \backslash neq\; \backslash mu\_2$) in a standard analysis, even though it is clear that the bivariate populations have different means. This illustration is based on $\backslash beta\_\{2\}\backslash neq\; 0$ to facilitate visualization; the same conclusions could be reached using a different linear combination of the elements of $\backslash beta$.

The maximum likelihood envelope estimator of $\backslash beta$, $\backslash hat\{\backslash beta\}\_\{\backslash text\{env\}\}$, however, can be formed by first projecting the data onto $\backslash mathcal\{E\}\_\{\backslash Sigma\}(\backslash mathcal\{B\})$ to remove the immaterial information, and then projecting onto horizontal axis, as shown by the paths marked "B" in the right panel. The two curves at the horizontal axis are the projection distribution from envelope analysis. The two populations have been arranged so that they have equal distribution when projected onto $\backslash mathcal\{E\}\_\backslash Sigma^\backslash bot(\backslash mathcal\{B\})$; that is, $Q\_\backslash varepsilon\; Y\backslash mid\; (X=0)\backslash sim\; Q\_\backslash varepsilon\; Y\backslash mid\; (X=1)$, but if the distribution is projected onto $\backslash mathcal\{E\}\_\backslash Sigma(\backslash mathcal\{B\})$, they have different distributions. The dashed line direction is the immaterial part that is in orthogonal complement of envelope subpace, and the solid line direction is material part of population that is the envelope subspace. From this figure, it is obvious that the two population means $\backslash mu\_1$ and $\backslash mu\_2$ differ significantly.

Another example is given in and deals with wheat protein data. This dataset contains measurements and the logarithms of near infrared reflectance at six wavelengths across the range 1680-2310 nm, measured on each of $n=50$ samples ground wheat. We take $r=2$ wavelengths as responses $Y=(Y\_1,Y\_2)^T$ and convert the continuous measure of protein into a categorical predictor $X$ indicating low and high protein (with sample sizes 24 and 26 respectively). The mean difference $\backslash mu\_1-\backslash mu\_2$ corresponds to the parameter vector $\backslash beta$ in the standard model with $X$ representing a binary indicator: $X=0$ for high protein, and $X=1$ for low protein wheat.

For this dataset, $\backslash hat\{\backslash beta\}\_\{\backslash text\{OLS\}\}^\{T\}=(7.5,-2.1)$, with standard errors $8.6$ and $9.5$ respectively. There is no indication from these marginal results that $Y$ depends on $X$, since the likelihood ratio test statistic has the value 27.5 on 2 degrees of freedom. Under a standard analysis, the simultaneous occurrence of relatively small $Z$-scores and a relatively large likelihood ratio statistic indicates that an envelope analysis offers advantages, although these conditions are certainly not necessary.

In particular, the envelope estimator is $\backslash hat\{\backslash beta\}\_\backslash text\{env\}^T=(5.1,-4.7)$, with standard errors of $.51$ and $.46$. We see that the standard errors for the envelope estimates of the coefficients are significantly lower than the standard errors for the OLS estimates, indicating substantial efficiency gains in estimation. To achieve the magnitude of this drop in standard errors under the standard model, we would need a sample size of $n\; \backslash approx\; 20,000$ to reduce the standard error from 9.4 to .46.

The envelope model has since been extended well beyond multivariate linear regression to many other areas in statistics. In 2011, Su and Cook introduced the partial envelope model{{cite journal | title=Partial Envelopes for Efficient Estimation in Multivariate Linear Regression| last=Su | first=Z. |author2=Cook, R.D.| journal=Biometrika | volume=98 | year=2011 | pages=133–146 | issue=1}} which allows us to apply the envelope method to a subset of predictors of interest.

In 2013, Cook et al. developed the predictor envelope{{cite journal | title= Envelopes and Partial Least Squares Regression

| last=Cook | first=R.D. |author2=Helland, I.S.|author3=Su, Z.| journal= Journal of the Royal Statistical Society: Series B (Statistical Methodology) | volume=75 | year=2013 | pages=851–877 | issue=5}} which applies the envelope method to the predictor space, $X\backslash in\backslash mathbb\{R\}^p$, where $X$ is stochastic with mean $\backslash mu\_\{X\}$ and covariance matrix $\backslash Sigma\_\{X\}$. In 2015, Cook et al. proposed the reduced-rank envelope model,{{cite journal | title= Envelopes and reduced-rank regression

| last=Cook | first=R.D. |author2=Forzani, L.|author3=Zhang, X.|journal= Biometrika | volume=102 | year=2015 | pages=439–456 | issue=2}} which combines the strength of reduced-rank regression and envelope model. Cook and Zhang{{cite journal | title= Foundations for envelope models and methods

| last=Cook | first=R.D. |author2=Zhang, X.| journal=Journal of the American Statistical Association | volume=110 | year=2015 | pages=599–611 | issue=510}} built the envelope model to generalized linear models such as weighted least squares, Cox regression, discriminant analysis, logistic regression, and Poisson regression.

The envelope model can also be applied to other contexts, such as variable selection, Bayesian inference, and high-dimensional data analysis. In 2016, Su et al. introduced the sparse envelope model{{cite journal | title= Sparse Envelope Model: Efficient Estimation and Response Variable Selection in Multivariate Linear Regression

| last=Su | first=Z. |author2=Zhu, G.|author3=Chen, X.|author4=Yang, X.| journal= Biometrika | volume=103 | year=2016 | pages=579–593 | issue=3}} which performs variable selection of the response variables, allowing us to identify the response variables for which the regression coefficients are zero. In 2017, Khare et al. developed the Bayesian envelope model{{cite journal | title= A Bayesian Approach for Envelope Models

| last=Khare | first=K. |author2=Pal, S.|author3=Su,Z.| journal= The Annals of Statistics | volume=45 | year=2017 | pages=196–222 | issue=1}} which allows us to incorporate prior information of the parameters into the analysis. The Bayesian envelope model is also applicable where the sample size is smaller than the number of responses.

The Matlab toolbox [https://github.com/emeryyi/envlp envlp ] and R package [http://www.stat.ufl.edu/~zhihuasu/styled-18/index.html Renvlp ] implement a variety of envelope estimators under the framework of multivariate linear regression, including the envelope model, the partial envelope model, and the predictor envelope. The capabilities of this toolbox include estimation of the model parameters, as well as performing standard multivariate inference in the context of envelope models; for example, hypothesis tests, and bootstrap. Examples and datasets are contained in the toolbox to illustrate the use of each model.{{cite journal | title=envlp: A MATLAB Toolbox for Computing Envelope Estimators in Multivariate Linear Regression

| last=Cook | first=R.D. |author2=Su, Z.|author3=Yang, Y.| journal=Journal of Statistical Software | volume=62 | year=2015 | pages=1–20 | issue=1}}

• Bivariate analysis
• Dimension reduction
• Functional data analysis
• Generalized linear model
• Linear discriminant analysis
• Maximum likelihood estimation
• Multivariate analysis
• Multivariate linear regression
• Ordinary least squares
• Partial least squares regression
• Regression analysis
• Statistical interference
• Sufficient dimension reduction

{{Reflist}}

{{CC-notice|cc=by3|url=https://www.jstatsoft.org/article/view/v062i08|author=R. Dennis Cook, Zhihua Su, Yi Yang}}

{{DEFAULTSORT:Linear Regression}}

tr:Regresyon analizi