Mercer's theorem#Mercer's condition

To explain Mercer's theorem, we first consider an important special case; see below for a more general formulation.

A kernel, in this context, is a symmetric continuous function

:$K:\; [a,b]\; \backslash times\; [a,b]\; \backslash rightarrow\; \backslash mathbb\{R\}$

where $K(x,y)\; =\; K(y,x)$ for all $x,y\; \backslash in\; [a,b]$.

K is said to be a positive-definite kernel if and only if

:$\backslash sum\_\{i=1\}^n\backslash sum\_\{j=1\}^n\; K(x\_i,\; x\_j)\; c\_i\; c\_j\; \backslash geq\; 0$

for all finite sequences of points x₁, ..., x_n of [a, b] and all choices of real numbers c₁, ..., c_n. Note that the term "positive-definite" is well-established in literature despite the weak inequality in the definition.{{Cite book |last=Mohri |first=Mehryar |url=https://www.worldcat.org/oclc/1041560990 |title=Foundations of machine learning |date=2018 |others=Afshin Rostamizadeh, Ameet Talwalkar |isbn=978-0-262-03940-6 |edition=Second |location=Cambridge, Massachusetts |oclc=1041560990}}{{Cite book |last=Berlinet |first=A. |url=https://www.worldcat.org/oclc/844346520 |title=Reproducing kernel Hilbert spaces in probability and statistics |date=2004 |publisher=Springer Science+Business Media |others=Christine Thomas-Agnan |isbn=1-4419-9096-8 |location=New York |oclc=844346520}}

The fundamental characterization of stationary positive-definite kernels (where $K(x,y)\; =\; K(x-y)$) is given by Bochner's theorem. It states that a continuous function $K(x-y)$ is positive-definite if and only if it can be expressed as the Fourier transform of a finite non-negative measure $\backslash mu$:

:$K(x-y)\; =\; \backslash int\_\{-\backslash infty\}^\{\backslash infty\}\; e^\{i(x-y)\backslash omega\}\; \backslash ,\; d\backslash mu(\backslash omega)$

This spectral representation reveals the connection between positive definiteness and harmonic analysis, providing a stronger and more direct characterization of positive definiteness than the abstract definition in terms of inequalities when the kernel is stationary, e.g, when it can be expressed as a 1-variable function of the distance between points rather than the 2-variable function of the positions of pairs of points.

Associated to K is a linear operator (more specifically a Hilbert–Schmidt integral operator when the interval is compact) on functions defined by the integral

:$[T\_K\; \backslash varphi](x)\; =\backslash int\_a^b\; K(x,s)\; \backslash varphi(s)\backslash ,\; ds.$

We assume $\backslash varphi$ can range through the space

of real-valued square-integrable functions L²[a, b]; however, in many cases the associated RKHS can be strictly larger than L²[a, b]. Since T_K is a linear operator, the eigenvalues and eigenfunctions of T_K exist.

Theorem. Suppose K is a continuous symmetric positive-definite kernel. Then there is an orthonormal basis

{e_i}_i of L²[a, b] consisting of eigenfunctions of T_K such that the corresponding

sequence of eigenvalues {λ_i}_i is nonnegative. The eigenfunctions corresponding to non-zero eigenvalues are continuous on [a, b] and K has the representation

:$K(s,t)\; =\; \backslash sum\_\{j=1\}^\backslash infty\; \backslash lambda\_j\; \backslash ,\; e\_j(s)\; \backslash ,\; e\_j(t)$

where the convergence is absolute and uniform.

We now explain in greater detail the structure of the proof of

Mercer's theorem, particularly how it relates to spectral theory of compact operators.

• The map K ↦ T_K is injective.
• T_K is a non-negative symmetric compact operator on L²[a,b]; moreover K(x, x) ≥ 0.

To show compactness, show that the image of the unit ball of L²[a,b] under T_K is equicontinuous and apply Ascoli's theorem, to show that the image of the unit ball is relatively compact in C([a,b]) with the uniform norm and a fortiori in L²[a,b].

Now apply the spectral theorem for compact operators on Hilbert

spaces to T_K to show the existence of the

orthonormal basis {e_i}_i of

L²[a,b]

:$\backslash lambda\_i\; e\_i(t)=\; [T\_K\; e\_i](t)\; =\; \backslash int\_a^b\; K(t,s)\; e\_i(s)\backslash ,\; ds.$

If λ_i ≠ 0, the eigenvector (eigenfunction) e_i is seen to be continuous on [a,b]. Now

:$\backslash sum\_\{i=1\}^\backslash infty\; \backslash lambda\_i\; |e\_i(t)\; e\_i(s)|\; \backslash leq\; \backslash sup\_\{x\; \backslash in\; [a,b]\}\; |K(x,x)|,$

which shows that the sequence

:$\backslash sum\_\{i=1\}^\backslash infty\; \backslash lambda\_i\; e\_i(t)\; e\_i(s)$

converges absolutely and uniformly to a kernel K₀ which is easily seen to define the same operator as the kernel K. Hence K=K₀ from which Mercer's theorem follows.

Finally, to show non-negativity of the eigenvalues one can write $\backslash lambda\; \backslash langle\; f,f\; \backslash rangle=\; \backslash langle\; f,\; T\_\{K\}f\; \backslash rangle$ and expressing the right hand side as an integral well-approximated by its Riemann sums, which are non-negative

by positive-definiteness of K, implying $\backslash lambda\; \backslash langle\; f,f\; \backslash rangle\; \backslash geq\; 0$, implying $\backslash lambda\; \backslash geq\; 0$.

The following is immediate:

Theorem. Suppose K is a continuous symmetric positive-definite kernel; T_K has a sequence of nonnegative

eigenvalues {λ_i}_i. Then

:$\backslash int\_a^b\; K(t,t)\backslash ,\; dt\; =\; \backslash sum\_i\; \backslash lambda\_i.$

This shows that the operator T_K is a trace class operator and

:$\backslash operatorname\{trace\}(T\_K)\; =\; \backslash int\_a^b\; K(t,t)\backslash ,\; dt.$

Mercer's theorem itself is a generalization of the result that any symmetric positive-semidefinite matrix is the Gramian matrix of a set of vectors.

The first generalization replaces the interval [a, b] with any compact Hausdorff space and Lebesgue measure on [a, b] is replaced by a finite countably additive measure μ on the Borel algebra of X whose support is X. This means that μ(U) > 0 for any nonempty open subset U of X.

A recent generalization replaces these conditions by the following: the set X is a first-countable topological space endowed with a Borel (complete) measure μ. X is the support of μ and, for all x in X, there is an open set U containing x and having finite measure. Then essentially the same result holds:

Theorem. Suppose K is a continuous symmetric positive-definite kernel on X. If the function κ is L¹_μ(X), where κ(x)=K(x,x), for all x in X, then there is an orthonormal set

{e_i}_i of L²_μ(X) consisting of eigenfunctions of T_K such that corresponding

sequence of eigenvalues {λ_i}_i is nonnegative. The eigenfunctions corresponding to non-zero eigenvalues are continuous on X and K has the representation

:$K(s,t)\; =\; \backslash sum\_\{j=1\}^\backslash infty\; \backslash lambda\_j\; \backslash ,\; e\_j(s)\; \backslash ,\; e\_j(t)$

where the convergence is absolute and uniform on compact subsets of X.

The next generalization deals with representations of measurable kernels.

Let (X, M, μ) be a σ-finite measure space. An L² (or square-integrable) kernel on X is a function

:$K\; \backslash in\; L^2\_\{\backslash mu\; \backslash otimes\; \backslash mu\}(X\; \backslash times\; X).$

L² kernels define a bounded operator T_K by the formula

:$\backslash langle\; T\_K\; \backslash varphi,\; \backslash psi\; \backslash rangle\; =\; \backslash int\_\{X\; \backslash times\; X\}\; K(y,x)\; \backslash varphi(y)\; \backslash psi(x)\; \backslash ,d[\backslash mu\; \backslash otimes\; \backslash mu](y,x).$

T_K is a compact operator (actually it is even a Hilbert–Schmidt operator). If the kernel K is symmetric, by the spectral theorem, T_K has an orthonormal basis of eigenvectors. Those eigenvectors that correspond to non-zero eigenvalues can be arranged in a sequence {e_i}_i (regardless of separability).

Theorem. If K is a symmetric positive-definite kernel on (X, M, μ), then

:$K(y,x)\; =\; \backslash sum\_\{i\; \backslash in\; \backslash mathbb\{N\}\}\; \backslash lambda\_i\; e\_i(y)\; e\_i(x)$

where the convergence in the L² norm. Note that when continuity of the kernel is not assumed, the expansion no longer converges uniformly.

In mathematics, a real-valued function K(x,y) is said to fulfill Mercer's condition if for all square-integrable functions g(x) one has

:$\backslash iint\; g(x)K(x,y)g(y)\backslash ,dx\backslash ,dy\; \backslash geq\; 0.$

This is analogous to the definition of a positive-semidefinite matrix. This is a matrix $K$ of dimension $N$, which satisfies, for all vectors $g$, the property

:$(g,Kg)=g^\{T\}\{\backslash cdot\}Kg=\backslash sum\_\{i=1\}^N\backslash sum\_\{j=1\}^N\backslash ,g\_i\backslash ,K\_\{ij\}\backslash ,g\_j\backslash geq0$.

A positive constant function

:$K(x,\; y)=c\backslash ,$

satisfies Mercer's condition, as then the integral becomes by Fubini's theorem

:$\backslash iint\; g(x)\backslash ,c\backslash ,g(y)\backslash ,dx\; \backslash ,\; dy\; =\; c\backslash int\backslash !\; g(x)\; \backslash ,dx\; \backslash int\backslash !\; g(y)\; \backslash ,dy\; =\; c\backslash left(\backslash int\backslash !\; g(x)\; \backslash ,dx\backslash right)^2$

which is indeed non-negative.

• Kernel trick
• Representer theorem
• Reproducing kernel Hilbert space
• Spectral theory

{{reflist}}

• Adriaan Zaanen, Linear Analysis, North Holland Publishing Co., 1960,
• Ferreira, J. C., Menegatto, V. A., Eigenvalues of integral operators defined by smooth positive definite kernels, Integral equation and Operator Theory, 64 (2009), no. 1, 61–81. (Gives the generalization of Mercer's theorem for metric spaces. The result is easily adapted to first countable topological spaces)
• Konrad Jörgens, Linear integral operators, Pitman, Boston, 1982,
• Richard Courant and David Hilbert, Methods of Mathematical Physics, vol 1, Interscience 1953,
• Robert Ash, Information Theory, Dover Publications, 1990,
• {{citation

|first=J. |last=Mercer

|title=Functions of positive and negative type and their connection with the theory of integral equations

|journal=Philosophical Transactions of the Royal Society A

|year=1909 |volume=209 |pages=415–446

|doi=10.1098/rsta.1909.0016

|issue=441–458

|bibcode=1909RSPTA.209..415M

|doi-access=free

}},

• {{springer|title=Mercer theorem|id=p/m063440}}
• H. König, Eigenvalue distribution of compact operators, Birkhäuser Verlag, 1986. (Gives the generalization of Mercer's theorem for finite measures μ.)

{{Functional analysis}}

Category:Theorems in functional analysis