Bayesian linear regression

Consider a standard linear regression problem, in which for $i\; =\; 1,\; \backslash ldots,\; n$ we specify the mean of the conditional distribution of $y\_i$ given a $k\; \backslash times\; 1$ predictor vector $\backslash mathbf\{x\}\_i$:

$$y\_\{i\}\; =\; \backslash mathbf\{x\}\_i^\backslash mathsf\{T\}\; \backslash boldsymbol\backslash beta\; +\; \backslash varepsilon\_i,$$

where $\backslash boldsymbol\backslash beta$ is a $k\; \backslash times\; 1$ vector, and the $\backslash varepsilon\_i$ are independent and identically normally distributed random variables:

$$\backslash varepsilon\_\{i\}\; \backslash sim\; N(0,\; \backslash sigma^2).$$

This corresponds to the following likelihood function:

$$\backslash rho(\backslash mathbf\{y\}\backslash mid\backslash mathbf\{X\},\backslash boldsymbol\backslash beta,\backslash sigma^\{2\})\; \backslash propto\; (\backslash sigma^2)^\{-n/2\}\; \backslash exp\backslash left(-\backslash frac\{1\}\{2\backslash sigma^2\}\; (\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash boldsymbol\backslash beta)^\backslash mathsf\{T\}(\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash boldsymbol\backslash beta)\backslash right).$$

The ordinary least squares solution is used to estimate the coefficient vector using the Moore–Penrose pseudoinverse:

$$\backslash hat\{\backslash boldsymbol\backslash beta\}\; =\; (\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\})^\{-1\}\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{y\}$$

where $\backslash mathbf\{X\}$ is the $n\; \backslash times\; k$ design matrix, each row of which is a predictor vector $\backslash mathbf\{x\}\_i^\backslash mathsf\{T\}$; and $\backslash mathbf\{y\}$ is the column $n$-vector $[y\_1\; \backslash ;\; \backslash cdots\; \backslash ;\; y\_n]^\backslash mathsf\{T\}$.

This is a frequentist approach, and it assumes that there are enough measurements to say something meaningful about $\backslash boldsymbol\backslash beta$. In the Bayesian approach,{{cite journal | last1 = Huang | first1 = Yunfei | last2 = Gompper | first2 = Gerhard | last3 = Sabass | first3 = Benedikt |year = 2020 | title = A Bayesian traction force microscopy method with automated denoising in a user-friendly software package | journal = Computer Physics Communications | volume = 256 | page = 107313 | doi = 10.1016/j.cpc.2020.107313 | arxiv = 2005.01377 | bibcode = 2020CoPhC.25607313H }} the data are supplemented with additional information in the form of a prior probability distribution. The prior belief about the parameters is combined with the data's likelihood function according to Bayes' theorem to yield the posterior belief about the parameters $\backslash boldsymbol\backslash beta$ and $\backslash sigma$. The prior can take different functional forms depending on the domain and the information that is available a priori.

Since the data comprise both $\backslash mathbf\{y\}$ and $\backslash mathbf\{X\}$, the focus only on the distribution of $\backslash mathbf\{y\}$ conditional on $\backslash mathbf\{X\}$ needs justification. In fact, a "full" Bayesian analysis would require a joint likelihood $\backslash rho(\backslash mathbf\{y\},\backslash mathbf\{X\}\backslash mid\backslash boldsymbol\backslash beta,\backslash sigma^\{2\},\backslash gamma)$ along with a prior $\backslash rho(\backslash beta,\backslash sigma^\{2\},\backslash gamma)$, where $\backslash gamma$ symbolizes the parameters of the distribution for $\backslash mathbf\{X\}$. Only under the assumption of (weak) exogeneity can the joint likelihood be factored into $\backslash rho(\backslash mathbf\{y\}\backslash mid\backslash boldsymbol\backslash mathbf\{X\},\backslash beta,\backslash sigma^\{2\})\backslash rho(\backslash mathbf\{X\}\backslash mid\backslash gamma)$.See Jackman (2009), p. 101. The latter part is usually ignored under the assumption of disjoint parameter sets. More so, under classic assumptions $\backslash mathbf\{X\}$ are considered chosen (for example, in a designed experiment) and therefore has a known probability without parameters.See Gelman et al. (2013), p. 354.

For an arbitrary prior distribution, there may be no analytical solution for the posterior distribution. In this section, we will consider a so-called conjugate prior for which the posterior distribution can be derived analytically.

A prior $\backslash rho(\backslash boldsymbol\backslash beta,\backslash sigma^\{2\})$ is conjugate to this likelihood function if it has the same functional form with respect to $\backslash boldsymbol\backslash beta$ and $\backslash sigma$. Since the log-likelihood is quadratic in $\backslash boldsymbol\backslash beta$, the log-likelihood is re-written such that the likelihood becomes normal in $(\backslash boldsymbol\backslash beta-\backslash hat\{\backslash boldsymbol\backslash beta\})$. Write

$$\backslash begin\{align\}$$

(\mathbf{y}- \mathbf{X} \boldsymbol\beta)^\mathsf{T}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)

&= [(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta}) + (\mathbf{X} \hat{\boldsymbol\beta} - \mathbf{X} \boldsymbol\beta)]^\mathsf{T} [(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta}) + (\mathbf{X} \hat{\boldsymbol\beta} - \mathbf{X} \boldsymbol\beta)] \\

&= (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})^\mathsf{T}(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta}) + (\boldsymbol\beta - \hat{\boldsymbol\beta})^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})(\boldsymbol\beta - \hat{\boldsymbol\beta})

+ \underbrace{2(\mathbf{X} \hat{\boldsymbol\beta} - \mathbf{X} \boldsymbol\beta)^\mathsf{T} (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})}_{= \ 0}\\

&= (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})^\mathsf{T}(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta}) + (\boldsymbol\beta - \hat{\boldsymbol\beta})^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})(\boldsymbol\beta - \hat{\boldsymbol\beta})\,.

\end{align}

The likelihood is now re-written as

$$\backslash rho(\backslash mathbf\{y\}|\backslash mathbf\{X\},\backslash boldsymbol\backslash beta,\backslash sigma^\{2\})\; \backslash propto\; (\backslash sigma^2)^\{-\backslash frac\{v\}\{2\}\}\; \backslash exp\backslash left(-\backslash frac\{vs^\{2\}\}\{2\{\backslash sigma\}^\{2\}\}\backslash right)(\backslash sigma^2)^\{-\backslash frac\{n-v\}\{2\}\}\; \backslash exp\backslash left(-\backslash frac\{1\}\{2\{\backslash sigma\}^\{2\}\}(\backslash boldsymbol\backslash beta\; -\; \backslash hat\{\backslash boldsymbol\backslash beta\})^\backslash mathsf\{T\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\})(\backslash boldsymbol\backslash beta\; -\; \backslash hat\{\backslash boldsymbol\backslash beta\})\backslash right),$$

where

$$vs^2\; =(\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash hat\{\backslash boldsymbol\backslash beta\})^\backslash mathsf\{T\}(\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash hat\{\backslash boldsymbol\backslash beta\})\; \backslash quad\; \backslash text\{\; and\; \}\; \backslash quad\; v\; =\; n-k,$$

where $k$ is the number of regression coefficients.

This suggests a form for the prior:

$$\backslash rho(\backslash boldsymbol\backslash beta,\backslash sigma^2)\; =\; \backslash rho(\backslash sigma^2)\backslash rho(\backslash boldsymbol\backslash beta\backslash mid\backslash sigma^2),$$

where $\backslash rho(\backslash sigma^2)$ is an inverse-gamma distribution

$$\backslash rho(\backslash sigma^2)\; \backslash propto\; (\backslash sigma^2)^\{-\backslash frac\{v\_0\}\{2\}-1\}\; \backslash exp\backslash left(-\backslash frac\{v\_0\; s\_0^2\}\{2\backslash sigma^2\}\backslash right).$$

In the notation introduced in the inverse-gamma distribution article, this is the density of an $\backslash text\{Inv-Gamma\}(\; a\_0,\; b\_0)$ distribution with $a\_0=\backslash tfrac\{v\_0\}\{2\}$ and $b\_0=\backslash tfrac\{1\}\{2\}\; v\_0s\_0^2$ with $v\_0$ and $s\_0^2$ as the prior values of $v$ and $s^\{2\}$, respectively. Equivalently, it can also be described as a scaled inverse chi-squared distribution, $\backslash text\{Scale-inv-\}\backslash chi^2(v\_0,\; s\_0^2).$

Further the conditional prior density $\backslash rho(\backslash boldsymbol\backslash beta|\backslash sigma^\{2\})$ is a normal distribution,

$$\backslash rho(\backslash boldsymbol\backslash beta\backslash mid\backslash sigma^2)\; \backslash propto\; (\backslash sigma^2)^\{-k/2\}\; \backslash exp\backslash left(-\backslash frac\{1\}\{2\backslash sigma^2\}(\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_0)^\backslash mathsf\{T\}\; \backslash mathbf\{\backslash Lambda\}\_0\; (\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_0)\backslash right).$$

In the notation of the normal distribution, the conditional prior distribution is $\backslash mathcal\{N\}\backslash left(\backslash boldsymbol\backslash mu\_0,\; \backslash sigma^2\; \backslash boldsymbol\backslash Lambda\_0^\{-1\}\backslash right).$

With the prior now specified, the posterior distribution can be expressed as

$$\backslash begin\{align\}$$

\rho(\boldsymbol\beta,\sigma^2\mid\mathbf{y},\mathbf{X}) &\propto \rho(\mathbf{y}\mid\mathbf{X},\boldsymbol\beta,\sigma^2)\rho(\boldsymbol\beta\mid\sigma^2)\rho(\sigma^2) \\

& \propto (\sigma^2)^{-n/2} \exp\left(-\frac{1}{2{\sigma}^2}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)^\mathsf{T}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)\right) (\sigma^2)^{-k/2} \exp\left(-\frac{1}{2\sigma^2}(\boldsymbol\beta -\boldsymbol\mu_0)^\mathsf{T} \boldsymbol\Lambda_0 (\boldsymbol\beta - \boldsymbol\mu_0)\right) (\sigma^2)^{-(a_0+1)} \exp\left(-\frac{b_0}{\sigma^2}\right)

\end{align}

With some re-arrangement,The intermediate steps of this computation can be found in O'Hagan (1994) at the beginning of the chapter on Linear models. the posterior can be re-written so that the posterior mean $\backslash boldsymbol\backslash mu\_n$ of the parameter vector $\backslash boldsymbol\backslash beta$ can be expressed in terms of the least squares estimator $\backslash hat\{\backslash boldsymbol\backslash beta\}$ and the prior mean $\backslash boldsymbol\backslash mu\_0$, with the strength of the prior indicated by the prior precision matrix $\backslash boldsymbol\backslash Lambda\_0$

$$\backslash boldsymbol\backslash mu\_n\; =\; (\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\}+\backslash boldsymbol\backslash Lambda\_0)^\{-1\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\; \backslash mathbf\{X\}\backslash hat\{\backslash boldsymbol\backslash beta\}+\backslash boldsymbol\backslash Lambda\_0\backslash boldsymbol\backslash mu\_0)\; .$$

To justify that $\backslash boldsymbol\backslash mu\_n$ is indeed the posterior mean, the quadratic terms in the exponential can be re-arranged as a quadratic form in $\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_n$.The intermediate steps are in Fahrmeir et al. (2009) on page 188.

$$(\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash boldsymbol\backslash beta)^\backslash mathsf\{T\}(\backslash mathbf\{y\}-\; \backslash mathbf\{X\}\; \backslash boldsymbol\backslash beta)\; +\; (\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_0)^\backslash mathsf\{T\}\backslash boldsymbol\backslash Lambda\_0(\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_0)\; =(\backslash boldsymbol\backslash beta-\backslash boldsymbol\backslash mu\_n)^\backslash mathsf\{T\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\}+\backslash boldsymbol\backslash Lambda\_0)(\backslash boldsymbol\backslash beta-\backslash boldsymbol\backslash mu\_n)+\backslash mathbf\{y\}^\backslash mathsf\{T\}\backslash mathbf\{y\}-\backslash boldsymbol\backslash mu\_n^\backslash mathsf\{T\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\}+\backslash boldsymbol\backslash Lambda\_0)\backslash boldsymbol\backslash mu\_n+\backslash boldsymbol\backslash mu\_0^\backslash mathsf\{T\}\; \backslash boldsymbol\backslash Lambda\_0\backslash boldsymbol\backslash mu\_0\; .$$

Now the posterior can be expressed as a normal distribution times an inverse-gamma distribution:

$$\backslash rho(\backslash boldsymbol\backslash beta,\backslash sigma^2\backslash mid\backslash mathbf\{y\},\backslash mathbf\{X\})\; \backslash propto\; (\backslash sigma^2)^\{-k/2\}\; \backslash exp\backslash left(-\backslash frac\{1\}\{2\{\backslash sigma\}^\{2\}\}(\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_n)^\backslash mathsf\{T\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\; \backslash mathbf\{X\}+\backslash mathbf\{\backslash Lambda\}\_0)(\backslash boldsymbol\backslash beta\; -\; \backslash boldsymbol\backslash mu\_n)\backslash right)\; (\backslash sigma^2)^\{-\backslash frac\{n+2a\_0\}\{2\}-1\}\; \backslash exp\backslash left(-\backslash frac\{2\; b\_0+\backslash mathbf\{y\}^\backslash mathsf\{T\}\backslash mathbf\{y\}-\backslash boldsymbol\backslash mu\_n^\backslash mathsf\{T\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\; \backslash mathbf\{X\}+\backslash boldsymbol\backslash Lambda\_0)\backslash boldsymbol\backslash mu\_n+\backslash boldsymbol\backslash mu\_0^\backslash mathsf\{T\}\; \backslash boldsymbol\backslash Lambda\_0\; \backslash boldsymbol\backslash mu\_0\}\{2\backslash sigma^2\}\backslash right)\; .$$

Therefore, the posterior distribution can be parametrized as follows.

$$\backslash rho(\backslash boldsymbol\backslash beta,\backslash sigma^2\backslash mid\backslash mathbf\{y\},\backslash mathbf\{X\})\; \backslash propto\; \backslash rho(\backslash boldsymbol\backslash beta\; \backslash mid\; \backslash sigma^2,\backslash mathbf\{y\},\backslash mathbf\{X\})\; \backslash rho(\backslash sigma^2\backslash mid\backslash mathbf\{y\},\backslash mathbf\{X\}),$$

where the two factors correspond to the densities of $\backslash mathcal\{N\}\backslash left(\; \backslash boldsymbol\backslash mu\_n,\; \backslash sigma^2\backslash boldsymbol\backslash Lambda\_n^\{-1\}\; \backslash right)\backslash ,$ and $\backslash text\{Inv-Gamma\}\backslash left(a\_n,b\_n\; \backslash right)$ distributions, with the parameters of these given by

$$\backslash boldsymbol\backslash Lambda\_n=(\backslash mathbf\{X\}^\backslash mathsf\{T\}\backslash mathbf\{X\}+\backslash mathbf\{\backslash Lambda\}\_0),\; \backslash quad\; \backslash boldsymbol\backslash mu\_n\; =\; (\backslash boldsymbol\backslash Lambda\_n)^\{-1\}(\backslash mathbf\{X\}^\backslash mathsf\{T\}\; \backslash mathbf\{X\}\; \backslash hat\{\backslash boldsymbol\backslash beta\}\; +\; \backslash boldsymbol\backslash Lambda\_0\; \backslash boldsymbol\backslash mu\_0)\; ,$$

$$a\_n=\; a\_0\; +\; \backslash frac\{n\}\{2\},\; \backslash qquad\; b\_n=b\_0+\backslash frac\{1\}\{2\}(\backslash mathbf\{y\}^\backslash mathsf\{T\}\; \backslash mathbf\{y\}\; +\; \backslash boldsymbol\backslash mu\_0^\backslash mathsf\{T\}\; \backslash boldsymbol\backslash Lambda\_0\backslash boldsymbol\backslash mu\_0-\backslash boldsymbol\backslash mu\_n^\backslash mathsf\{T\}\; \backslash boldsymbol\backslash Lambda\_n\; \backslash boldsymbol\backslash mu\_n)\; .$$

which illustrates Bayesian inference being a compromise between the information contained in the prior and the information contained in the sample.

The model evidence $p(\backslash mathbf\{y\}\backslash mid\; m)$ is the probability of the data given the model $m$. It is also known as the marginal likelihood, and as the prior predictive density. Here, the model is defined by the likelihood function $p(\backslash mathbf\{y\}\backslash mid\backslash mathbf\{X\},\backslash boldsymbol\backslash beta,\backslash sigma)$ and the prior distribution on the parameters, i.e. $p(\backslash boldsymbol\backslash beta,\backslash sigma)$. The model evidence captures in a single number how well such a model explains the observations. The model evidence of the Bayesian linear regression model presented in this section can be used to compare competing linear models by Bayes factors. These models may differ in the number and values of the predictor variables as well as in their priors on the model parameters. Model complexity is already taken into account by the model evidence, because it marginalizes out the parameters by integrating $p(\backslash mathbf\{y\},\backslash boldsymbol\backslash beta,\backslash sigma\backslash mid\backslash mathbf\{X\})$ over all possible values of $\backslash boldsymbol\backslash beta$ and $\backslash sigma$.

$$p(\backslash mathbf\{y\}|m)=\backslash int\; p(\backslash mathbf\{y\}\backslash mid\backslash mathbf\{X\},\backslash boldsymbol\backslash beta,\backslash sigma)\backslash ,\; p(\backslash boldsymbol\backslash beta,\backslash sigma)\backslash ,\; d\backslash boldsymbol\backslash beta\backslash ,\; d\backslash sigma$$

This integral can be computed analytically and the solution is given in the following equation.The intermediate steps of this computation can be found in O'Hagan (1994) on page 257.

$$p(\backslash mathbf\{y\}\backslash mid\; m)=\backslash frac\{1\}\{(2\backslash pi)^\{n/2\}\}\backslash sqrt\{\backslash frac\{\backslash det(\backslash boldsymbol\backslash Lambda\_0)\}\{\backslash det(\backslash boldsymbol\backslash Lambda\_n)\}\}\; \backslash cdot\; \backslash frac\{b\_0^\{a\_0\}\}\{b\_n^\{a\_n\}\}\; \backslash cdot\; \backslash frac\{\backslash Gamma(a\_n)\}\{\backslash Gamma(a\_0)\}$$

Here $\backslash Gamma$ denotes the gamma function. Because we have chosen a conjugate prior, the marginal likelihood can also be easily computed by evaluating the following equality for arbitrary values of $\backslash boldsymbol\backslash beta$ and $\backslash sigma$.{{cite journal |last=Chib |first=Siddhartha |year=1995 |title=Marginal Likelihood from the Gibbs Output |journal=Journal of the American Statistical Association |volume=90 |issue=432 |pages=1313–1321 |doi=10.2307/2291521 }}

$$p(\backslash mathbf\{y\}\backslash mid\; m)=\backslash frac\{p(\backslash boldsymbol\backslash beta,\backslash sigma|m)\backslash ,\; p(\backslash mathbf\{y\}\; \backslash mid\; \backslash mathbf\{X\},\; \backslash boldsymbol\backslash beta,\backslash sigma,m)\}\{p(\backslash boldsymbol\backslash beta,\; \backslash sigma\; \backslash mid\; \backslash mathbf\{y\},\backslash mathbf\{X\},m)\}$$

Note that this equation follows from a re-arrangement of Bayes' theorem. Inserting the formulas for the prior, the likelihood, and the posterior and simplifying the resulting expression leads to the analytic expression given above.

In general, it may be impossible or impractical to derive the posterior distribution analytically. However, it is possible to approximate the posterior by an approximate Bayesian inference method such as Monte Carlo sampling,Carlin and Louis (2008) and Gelman, et al. (2003) explain how to use sampling methods for Bayesian linear regression. INLA or variational Bayes.

The special case $\backslash boldsymbol\backslash mu\_0=0,\; \backslash mathbf\{\backslash Lambda\}\_0\; =\; c\backslash mathbf\{I\}$ is called ridge regression.

A similar analysis can be performed for the general case of the multivariate regression and part of this provides for Bayesian estimation of covariance matrices: see Bayesian multivariate linear regression.

• Bayes linear statistics
• Constrained least squares
• Regularized least squares
• Tikhonov regularization
• Spike and slab variable selection
• Bayesian interpretation of kernel regularization

{{More footnotes|date=August 2011}}

{{Reflist}}

• {{cite book |author-link1=George E. P. Box |last1=Box |first1=G. E. P. |author2-link=George Tiao|last2=Tiao |first2=G. C. |year=1973 |title=Bayesian Inference in Statistical Analysis |publisher=Wiley |isbn=0-471-57428-7 }}
• {{cite book|author=Carlin, Bradley P. |author2=Louis, Thomas A.| title = Bayesian Methods for Data Analysis |edition=Third | publisher = Boca Raton, FL: Chapman and Hall/CRC | year = 2008 | isbn = 978-1-58488-697-6}}
• {{cite book|author=Fahrmeir, L. |author2=Kneib, T. |author3=Lang, S.| title = Regression. Modelle, Methoden und Anwendungen |edition=Second |publisher = Springer |location=Heidelberg | year = 2009 | isbn = 978-3-642-01836-7 | doi = 10.1007/978-3-642-01837-4}}
• {{cite book |last1=Gelman |first1=Andrew |author-link1=Andrew Gelman |last2=Carlin |first2=John B. |last3=Stern |first3=Hal S. |last4=Dunson |first4=David B. |last5=Vehtari |first5=Aki |last6=Rubin |first6=Donald B. |display-authors=1 |chapter=Introduction to regression models |pages=353–380 |title = Bayesian Data Analysis |edition=Third | publisher = Boca Raton, FL: Chapman and Hall/CRC | year = 2013 | isbn = 978-1-4398-4095-5 }}
• {{cite book |first=Simon |last=Jackman |chapter=Regression models |pages=99–124 |title=Bayesian Analysis for the Social Sciences |location= |publisher=Wiley |year=2009 |isbn=978-0-470-01154-6 }}
• {{cite book |first1=Peter E. |last1=Rossi |first2=Greg M. |last2=Allenby |first3=Robert |last3=McCulloch |title=Bayesian Statistics and Marketing |publisher=John Wiley & Sons |year=2006 |isbn=0470863676 }}
• {{cite book|author=O'Hagan, Anthony| title = Bayesian Inference| volume= 2B |series = Kendall's Advanced Theory of Statistics| year = 1994 | edition= First | isbn = 0-340-52922-9| publisher = Halsted}}

• Bayesian estimation of linear models (R programming wikibook). Bayesian linear regression as implemented in R.

{{Least Squares and Regression Analysis}}

{{Statistics|correlation}}

Linear regression

Category:Single-equation methods (econometrics)