semiparametric regression

Many different semiparametric regression methods have been proposed and developed. The most popular methods are the partially linear, index and varying coefficient models.

A partially linear model is given by

: $Y\_i\; =\; X\text{'}\_i\; \backslash beta\; +\; g\backslash left(Z\_i\; \backslash right)\; +\; u\_i,\; \backslash ,\; \backslash quad\; i\; =\; 1,\backslash ldots,n,\; \backslash ,$

where $Y\_\{i\}$ is the dependent variable, $X\_\{i\}$ is a $p\; \backslash times\; 1$ vector of explanatory variables, $\backslash beta$ is a $p\; \backslash times\; 1$ vector of unknown parameters and $Z\_\{i\}\; \backslash in\; \backslash operatorname\{R\}^\{q\}$. The parametric part of the partially linear model is given by the parameter vector $\backslash beta$ while the nonparametric part is the unknown function $g\backslash left(Z\_\{i\}\backslash right)$. The data is assumed to be i.i.d. with $E\backslash left(u\_\{i\}|X\_\{i\},Z\_\{i\}\backslash right)\; =\; 0$ and the model allows for a conditionally heteroskedastic error process $E\backslash left(u^\{2\}\_\{i\}|x,z\backslash right)\; =\; \backslash sigma^\{2\}\backslash left(x,z\backslash right)$ of unknown form. This type of model was proposed by Robinson (1988) and extended to handle categorical covariates by Racine and Li (2007).

This method is implemented by obtaining a $\backslash sqrt\{n\}$ consistent estimator of $\backslash beta$ and then deriving an estimator of $g\backslash left(Z\_\{i\}\backslash right)$ from the nonparametric regression of $Y\_\{i\}\; -\; X\text{'}\_\{i\}\backslash hat\{\backslash beta\}$ on $z$ using an appropriate nonparametric regression method.See Li and Racine (2007) for an in-depth look at nonparametric regression methods.

A single index model takes the form

: $Y\; =\; g\backslash left(X\text{'}\backslash beta\_\{0\}\backslash right)\; +\; u,\; \backslash ,$

where $Y$, $X$ and $\backslash beta\_\{0\}$ are defined as earlier and the error term $u$ satisfies $E\backslash left(u|X\backslash right)\; =\; 0$. The single index model takes its name from the parametric part of the model $x\text{'}\backslash beta$ which is a scalar single index. The nonparametric part is the unknown function $g\backslash left(\backslash cdot\backslash right)$.

The single index model method developed by Ichimura (1993) is as follows. Consider the situation in which $y$ is continuous. Given a known form for the function $g\backslash left(\backslash cdot\backslash right)$, $\backslash beta\_\{0\}$ could be estimated using the nonlinear least squares method to minimize the function

: $\backslash sum\_\{i=1\}\; \backslash left(Y\_i\; -\; g\backslash left(X\text{'}\_i\; \backslash beta\backslash right)\backslash right)^2.$

Since the functional form of $g\backslash left(\backslash cdot\backslash right)$ is not known, we need to estimate it. For a given value for $\backslash beta$ an estimate of the function

: $G\backslash left(X\text{'}\_i\; \backslash beta\; \backslash right)\; =\; E\backslash left(Y\_i\; |X\text{'}\_i\; \backslash beta\backslash right)\; =\; E\backslash left[g\backslash left(X\text{'}\_i\backslash beta\_o\; \backslash right)|X\text{'}\_i\; \backslash beta\backslash right]$

using kernel method. Ichimura (1993) proposes estimating $g\backslash left(X\text{'}\_\{i\}\backslash beta\backslash right)$ with

: $\backslash hat\{G\}\_\{-i\}\backslash left(X\text{'}\_i\; \backslash beta\backslash right),\backslash ,$

the leave-one-out nonparametric kernel estimator of $G\backslash left(X\text{'}\_\{i\}\backslash beta\backslash right)$.

If the dependent variable $y$ is binary and $X\_\{i\}$ and $u\_\{i\}$ are assumed to be independent, Klein and Spady (1993) propose a technique for estimating $\backslash beta$ using maximum likelihood methods. The log-likelihood function is given by

: $L\backslash left(\backslash beta\backslash right)\; =\; \backslash sum\_i\; \backslash left(1-Y\_i\backslash right)\backslash ln\backslash left(1-\backslash hat\{g\}\_\{-i\}\backslash left(X\text{'}\_i\backslash beta\backslash right)\backslash right)\; +\; \backslash sum\_\{i\}Y\_i\backslash ln\backslash left(\backslash hat\{g\}\_\{-i\}\backslash left(X\text{'}\_i\; \backslash beta\backslash right)\backslash right),$

where $\backslash hat\{g\}\_\{-i\}\backslash left(X\text{'}\_\{i\}\backslash beta\backslash right)$ is the leave-one-out estimator.

Hastie and Tibshirani (1993) propose a smooth coefficient model given by

: $Y\_i\; =\; \backslash alpha\backslash left(Z\_i\backslash right)\; +\; X\text{'}\_i\backslash beta\backslash left(Z\_i\backslash right)\; +\; u\_i$

= \left(1 + X'_i\right)\left(\begin{array}{c} \alpha\left(Z_i\right) \\ \beta\left(Z_i\right) \end{array}\right) + u_i

= W'_i\gamma\left(Z_i\right) + u_i,

where $X\_\{i\}$ is a $k\; \backslash times\; 1$ vector and $\backslash beta\backslash left(z\backslash right)$ is a vector of unspecified smooth functions of $z$.

$\backslash gamma\backslash left(\backslash cdot\backslash right)$ may be expressed as

: $\backslash gamma\backslash left(Z\_i\backslash right)\; =\; \backslash left(E\backslash left[W\_i\; W\text{'}\_i|Z\_i\; \backslash right]\backslash right)^\{-1\}E\backslash left[W\_i\; Y\_i|Z\_i\backslash right].$

• Nonparametric regression
• Effective degree of freedom

{{reflist}}

• {{cite journal

| last = Robinson

| first = P.M.

| title = Root-n Consistent Semiparametric Regression

| journal = Econometrica

| volume = 56

| issue = 4

| pages = 931–954

| year = 1988

| doi =10.2307/1912705

| jstor = 1912705

| publisher = The Econometric Society

}}

• {{cite book

| last = Li

| first = Qi

|author2=Racine, Jeffrey S.

| title = Nonparametric Econometrics: Theory and Practice

| publisher = Princeton University Press

| year = 2007

| isbn = 978-0-691-12161-1

}}

• {{cite journal

| last = Racine

| first = J.S.

|author2=Qui, L.

| title = A Partially Linear Kernel Estimator for Categorical Data

| journal = Unpublished Manuscript, Mcmaster University

| year = 2007

}}

• {{cite journal

| last = Ichimura

| first = H.

| title = Semiparametric Least Squares (SLS) and Weighted SLS Estimation of Single Index Models

| journal = Journal of Econometrics

| volume = 58

| issue = 1–2

| pages = 71–120

| year = 1993

| doi =10.1016/0304-4076(93)90114-K

| url = http://purl.umn.edu/55563

}}

• {{cite journal

| last = Klein

| first = R. W.

|author2=R. H. Spady

| title = An Efficient Semiparametric Estimator for Binary Response Models

| journal = Econometrica

| volume = 61

| issue = 2

| pages = 387–421

| year = 1993

| doi =10.2307/2951556

| jstor = 2951556

| publisher = The Econometric Society

| citeseerx = 10.1.1.318.4925

}}

• {{cite journal

| last = Hastie

| first = T.

|author2=R. Tibshirani

| title = Varying-Coefficient Models

| journal = Journal of the Royal Statistical Society, Series B

| volume = 55

| pages = 757–796

| year = 1993

}}

{{Statistics|correlation}}

Category:Nonparametric statistics