Generalized filtering

Definition: Generalized filtering rests on the tuple $(\backslash Omega,U,X,S,p,q)$:

• A sample space $\backslash Omega$ from which random fluctuations $\backslash omega\; \backslash in\; \backslash Omega$ are drawn
• Control states $U\; \backslash in\; \backslash mathbb\{R\}$ – that act as external causes, input or forcing terms
• Hidden states $X:X\; \backslash times\; U\; \backslash times\; \backslash Omega\; \backslash to\; \backslash mathbb\{R\}$ – that cause sensory states and depend on control states
• Sensor states $S:X\; \backslash times\; U\; \backslash times\; \backslash Omega\; \backslash to\; \backslash mathbb\{R\}$ – a probabilistic mapping from hidden and control states
• Generative density $p(\backslash tilde\{s\},\backslash tilde\{x\},\backslash tilde\{u\}\backslash mid\; m)$ – over sensory, hidden and control states under a generative model $m$
• Variational density $q(\backslash tilde\{x\},\backslash tilde\{u\}\backslash mid\; \backslash tilde\{\backslash mu\})$ – over hidden and control states with mean $\backslash tilde\{\backslash mu\}\; \backslash in\; \backslash mathbb\{R\}$

Here ~ denotes a variable in generalized coordinates of motion: $\backslash tilde\{u\}\; =\; [u,u\text{'},u\text{'}\text{'},\backslash ldots]^T$

The objective is to approximate the posterior density over hidden and control states, given sensor states and a generative model – and estimate the (path integral of) model evidence $p(\backslash tilde\{s\}(t)\backslash vert\; m)$ to compare different models. This generally involves an intractable marginalization over hidden states, so model evidence (or marginal likelihood) is replaced with a variational free energy bound.R P Feynman, Statistical mechanics. Reading MA: Benjamin, 1972 Given the following definitions:

: $\backslash tilde\{\backslash mu\}(t)\; =\; \backslash underset\{\backslash tilde\{\backslash mu\}\}\{\backslash operatorname\{arg\backslash ,min\}\}\; \backslash \{F(\backslash tilde\{\{s\}\}(t),\backslash tilde\{\{\backslash mu\; \}\})\backslash \}$

: $G(\backslash tilde\{s\},\backslash tilde\{x\},\backslash tilde\{u\})=-\backslash ln\; p(\backslash tilde\{s\},\backslash tilde\{x\},\backslash tilde\{u\}\backslash vert\; m)$

Denote the Shannon entropy of the density $q$ by $H[q]=E\_q[-\backslash log(q)]$. We can then write the variational free energy in two ways:

: $F(\backslash tilde\{s\},\; \backslash tilde\backslash mu)=E\_q\; [G(\backslash tilde\{s\},\backslash tilde\{x\},\backslash tilde\{u\})]-H[q(\backslash tilde\{x\},\backslash tilde\{u\}\; \backslash vert\; \backslash tilde\{\backslash mu\})]\; =-\backslash ln\; p(\backslash tilde\{s\}\; \backslash vert\; m)+D\_\{KL\}\; [q(\backslash tilde\{x\},\backslash tilde\{u\}\; \backslash vert\; \backslash tilde\{\backslash mu\})\backslash vert\; \backslash vert\; p(\backslash tilde\{x\},\backslash tilde\{u\}\; \backslash vert\; \backslash tilde\{s\},m)]$

The second equality shows that minimizing variational free energy (i) minimizes the Kullback-Leibler divergence between the variational and true posterior density and (ii) renders the variational free energy (a bound approximation to) the negative log evidence (because the divergence can never be less than zero).M J Beal, "[http://www.cse.buffalo.edu/faculty/mbeal/papers/beal03.pdf Variational Algorithms for Approximate Bayesian Inference]," PhD. Thesis, University College London, 2003. Under the Laplace assumption $q(\backslash tilde\{x\},\backslash tilde\{\{u\}\}\backslash mid\; \backslash tilde\{\backslash mu\})=\backslash mathcal\{N\}(\backslash tilde\{\backslash mu\},C)$ the variational density is Gaussian and the precision that minimizes free energy is $C^\{-1\}=\backslash Pi\; =\backslash partial\_\{\backslash tilde\{\backslash mu\}\backslash tilde\{\backslash mu\}\}\; G(\backslash tilde\{\backslash mu\})$. This means that free-energy can be expressed in terms of the variational mean K Friston, J Mattout, N Trujillo-Barreto, J Ashburner, and W Penny, "[http://www.fil.ion.ucl.ac.uk/~karl/Variational%20free%20energy%20and%20the%20Laplace%20approximation.pdf Variational free energy and the Laplace approximation]," NeuroImage, vol. 34, no. 1, pp. 220-34, 2007 (omitting constants):

: $F=G(\backslash tilde\{\backslash mu\})+\backslash textstyle\{1\; \backslash over\; 2\}\backslash ln\; \backslash vert\; \backslash partial\_\{\backslash tilde\{\backslash mu\}\backslash tilde\{\backslash mu\}\}\; G(\backslash tilde\{\backslash mu\})\backslash vert$

The variational means that minimize the (path integral) of free energy can now be recovered by solving the generalized filter:

: $\backslash dot\{\backslash tilde\{\backslash mu\}\}=D\backslash tilde\{\backslash mu\}-\backslash partial\_\{\backslash tilde\{\backslash mu\}\}\; F(\backslash tilde\{s\},\backslash tilde\{\backslash mu\})$

where $D$ is a block matrix derivative operator of identify matrices such that $D\backslash tilde\{u\}=[u\text{'},u\text{'}\text{'},\backslash ldots\; ]^T$

Generalized filtering is based on the following lemma: The self-consistent solution to $\backslash dot\{\backslash tilde\{\backslash mu\}\}=D\backslash tilde\{\backslash mu\}-\backslash partial\_\{\backslash tilde\{\backslash mu\}\}\; F(s,\backslash tilde\{\backslash mu\})$ satisfies the variational principle of stationary action, where action is the path integral of variational free energy

: $S=\backslash int\; dt\backslash ,\; F(\backslash tilde\{s\}(t),\backslash tilde\{\backslash mu\}(t))$

Proof: self-consistency requires the motion of the mean to be the mean of the motion and (by the fundamental lemma of variational calculus)

: $\backslash dot\{\backslash tilde\{\backslash mu\}\}\; =\; D\backslash tilde\{\backslash mu\}\backslash Leftrightarrow\; \backslash partial\_\{\backslash tilde\{\backslash mu\}\}\; F(\backslash tilde\{s\},\backslash tilde\{\backslash mu\})\; =\; 0\backslash Leftrightarrow\; \backslash delta\_\{\backslash tilde\{\backslash mu\}\}\; S=0$

Put simply, small perturbations to the path of the mean do not change variational free energy and it has the least action of all possible (local) paths.

Remarks: Heuristically, generalized filtering performs a gradient descent on variational free energy in a moving frame of reference: $\backslash dot\{\{\backslash tilde\{\{\backslash mu\; \}\}\}\}-D\backslash tilde\{\{\backslash mu\; \}\}=-\backslash partial\_\{\backslash tilde\{\backslash mu\}\}\; F(s,\backslash tilde\{\backslash mu\})$, where the frame itself minimizes variational free energy. For a related example in statistical physics, see Kerr and Graham W C Kerr and A J Graham, "[http://epjb.epj.org/articles/epjb/abs/2000/10/b9689/b9689.html Generalised phase space version of Langevin equations and associated Fokker-Planck equations]," Eur. Phys. J. B., vol. 15, pp. 305-11, 2000. who use ensemble dynamics in generalized coordinates to provide a generalized phase-space version of Langevin and associated Fokker-Planck equations.

In practice, generalized filtering uses local linearizationT Ozaki, "[http://www3.stat.sinica.edu.tw/statistica/oldpdf/A2n16.pdf A bridge between nonlinear time-series models and nonlinear stochastic dynamical systems: A local linearization approach]," Statistica Sin., vol. 2, pp. 113-135, 1992 over intervals $\backslash Delta\; t$ to recover discrete updates

: $\backslash begin\{align\}$

\Delta \tilde{\mu} & =(\exp (\Delta t\cdot J)-I)J^{-1}\dot{\tilde{\mu}} \\

J & =\partial_{\tilde{\mu}} \dot{\tilde{\mu}} =D-\partial _{\tilde{\mu}\tilde{\mu}} F(\tilde{s},\tilde{\mu})

\end{align}

This updates the means of hidden variables at each interval (usually the interval between observations).

Usually, the generative density or model is specified in terms of a nonlinear input-state-output model with continuous nonlinear functions:

: $\backslash begin\{align\}$

s & = g(x,u)+\omega_s \\

\dot{{x}} & = f(x,u)+\omega_x

\end{align}

The corresponding generalized model (under local linearity assumptions) obtains the from the chain rule

: $$

\begin{align}

\tilde{s} & =\tilde{g}(\tilde{x},\tilde{u})+\tilde{\omega}_s \\

\\

s & =g(x,u)+\omega_s \\

s' & =\partial_x g\cdot x'+\partial_u g\cdot u'+\omega'_s \\

s & =\partial_x g\cdot x+\partial_u g\cdot u+\omega_s

\\

& \vdots \\

\end{align} \qquad \begin{align}

\dot{\tilde{x}} & =\tilde{f}(\tilde{x},\tilde{u})+\tilde{\omega}_x \\

\\

\dot{x} & =f(x,u)+\omega_x \\

\dot{x}' & =\partial_x f\cdot x'+\partial_u f\cdot u'+\omega'_x \\

\dot{x}& =\partial_x f\cdot x+\partial_u f\cdot u+\omega_x \\

& \vdots

\end{align}

Gaussian assumptions about the random fluctuations $\backslash omega$ then prescribe the likelihood and empirical priors on the motion of hidden states

: $\backslash begin\{align\}$

p\left( \tilde{s},\tilde{x},\tilde{u}\vert m \right) & = p\left( \tilde{s}\vert \tilde{x},\tilde{u},m \right)p\left( {D\tilde{x}\vert x,\tilde{u},m} \right)p(x\vert m)p(\tilde{u}\vert m) \\

p\left( \tilde{s}\vert \tilde{x},\tilde{u},m \right) & = \mathcal{N}(\tilde{g}(\tilde{x},\tilde{u}),\tilde{\Sigma}(\tilde{x},\tilde{u})_s) \\

p\left( {D\tilde{{x}}\vert x,\tilde{{u}},m} \right) & = {\mathcal{N}}(\tilde{f}(\tilde{x},\tilde{u}),\tilde{\Sigma}(\tilde{x},\tilde{u})_x ) \\

\end{align}

The covariances $\backslash tilde\{\{\backslash Sigma\; \}\}=V\backslash otimes\; \backslash Sigma$ factorize into a covariance among variables and correlations $V$ among generalized fluctuations that encodes their autocorrelation:

: $V=\backslash begin\{bmatrix\}$

1 & 0 & \ddot{\rho}(0) & \cdots \\

0 & -\ddot{\rho}(0) & 0 \ & \ \\

\ddot{\rho}(0) \ & 0 \ & \ddot{\ddot{\rho}}(0) \ & \ \\

\vdots \ & \ & \ & \ddots \ \\

\end{bmatrix}

Here, $\backslash ddot\{\{\backslash rho\; \}\}(0)$ is the second derivative of the autocorrelation function evaluated at zero. This is a ubiquitous measure of roughness in the theory of stochastic processes.D R Cox and H D Miller, The theory of stochastic processes. London: Methuen, 1965. Crucially, the precision (inverse variance) of high order derivatives fall to zero fairly quickly, which means it is only necessary to model relatively low order generalized motion (usually between two and eight) for any given or parameterized autocorrelation function.

When time series are observed as a discrete sequence of $N$ observations, the implicit sampling is treated as part of the generative process, where (using Taylor's theorem)

: $[s\_1\; ,\backslash dots\; ,s\_N\; ]^T\; =\; (E\backslash otimes\; I)\backslash cdot\; \backslash tilde\{s\}(t):\; \backslash qquad\; E\_\{ij\}\; =\backslash frac\{(i-t)^\{(j-1)\}\}\{(j-1)!\}$

In principle, the entire sequence could be used to estimate hidden variables at each point in time. However, the precision of samples in the past and future falls quickly and can be ignored. This allows the scheme to assimilate data online, using local observations around each time point (typically between two and eight).

For any slowly varying model parameters of the equations of motion $f(x,u,\backslash theta\; )$ or precision $\backslash tilde\{\{\backslash Pi\; \}\}(x,u,\backslash theta\; )$ generalized filtering takes the following form (where $\backslash mu$ corresponds to the variational mean of the parameters)

: $\backslash begin\{align\}$

\dot{\mu} & = \mu' \\

\dot{\mu'} & = -\partial_\mu F(\tilde{s},\mu )-\kappa \mu'

\end{align}

Here, the solution $\backslash dot\{\{\backslash tilde\{\{\backslash mu\; \}\}\}\}=0$ minimizes variational free energy, when the motion of the mean is small. This can be seen by noting $\backslash dot\{\{\backslash mu\; \}\}=\{\backslash dot\{\{\backslash mu\; \}\}\}\text{'}=0\backslash Rightarrow\; \backslash partial\_\{\backslash mu\; \}\; F=0\backslash Rightarrow\; \backslash delta\_\{\backslash mu\; \}\; S=0$. It is straightforward to show that this solution corresponds to a classical Newton update.K Friston, K Stephan, B Li, and J. Daunizeau, "Generalised Filtering," Mathematical Problems in Engineering, vol. vol., 2010, p. 621670, 2010.

Classical filtering under Markovian or Wiener assumptions is equivalent to assuming the precision of the motion of random fluctuations is zero. In this limiting case, one only has to consider the states and their first derivative $\backslash tilde\{\{\backslash mu\; \}\}=(\backslash mu\; ,\{\backslash mu\; \}\text{'})$. This means generalized filtering takes the form of a Kalman-Bucy filter, with prediction and correction terms:

: $\backslash begin\{align\}$

\dot{\mu} & = \mu'-\partial_\mu F(s,\tilde{\mu}) \\

\dot{\mu'} & =-\partial_{\mu'} F(s,\tilde{\mu})

\end{align}

Substituting this first-order filtering into the discrete update scheme above gives the equivalent of (extended) Kalman filtering.K J Friston, N Trujillo-Barreto, and J Daunizeau, "DEM: A variational treatment of dynamic systems," Neuroimage, vol. 41, no. 3, pp. 849-85, 2008

Particle filtering is a sampling-based scheme that relaxes assumptions about the form of the variational or approximate posterior density. The corresponding generalized filtering scheme is called variational filtering.K J Friston, "[http://www.fil.ion.ucl.ac.uk/~karl/Variational%20filtering.pdf Variational filtering]," Neuroimage, vol. 41, no. 3, pp. 747-66, 2008. In variational filtering, an ensemble of particles diffuse over the free energy landscape in a frame of reference that moves with the expected (generalized) motion of the ensemble. This provides a relatively simple scheme that eschews Gaussian (unimodal) assumptions. Unlike particle filtering it does not require proposal densities—or the elimination or creation of particles.

Variational Bayes rests on a mean field partition of the variational density:

: $q(\backslash tilde\{x\},\backslash tilde\{u\},\backslash theta\; \backslash dots\; \backslash vert\; \backslash tilde\{\backslash mu\},\backslash mu$

)=q(\tilde{x},\tilde{u}\vert \tilde{\mu})q(\theta \vert \mu

)\dots

This partition induces a variational update or step for each marginal density—that is usually solved analytically using conjugate priors. In generalized filtering, this leads to dynamic expectation maximisation.K J Friston, N Trujillo-Barreto, and J Daunizeau, "[http://www.fil.ion.ucl.ac.uk/~karl/DEM%20A%20variational%20treatment%20of%20dynamic%20systems.pdf DEM: A variational treatment of dynamic systems]," Neuroimage, vol. 41, no. 3, pp. 849-85, 2008 that comprises a D-step that optimizes the sufficient statistics of unknown states, an E-step for parameters and an M-step for precisions.

Generalized filtering is usually used to invert hierarchical models of the following form

: $\backslash begin\{align\}$

\tilde{s} & =\tilde{{g}}^1 (\tilde{x}^1,\tilde{u}^{(1)})+\tilde{\omega}_s^{(1)} \\

\dot{\tilde{x}}^{(1)} & =\tilde{f}^{(1)}(\tilde{x}^{(1)},\tilde{u}^{(1)})+\tilde{\omega}_{x}^{(1)} \\

\vdots \\

\tilde{u}^{(i-1)} & =\tilde{g}^{(i)}(\tilde{x}^{(i)},\tilde{u}^{(i)})+\tilde{\omega}_u^{(i)} \\

\dot{\tilde{x}}^{(i)} & =\tilde{f}^{(i)}(\tilde{x}^{(i)},\tilde{u}^{(i)})+\tilde{\omega}_x^{(i)} \\

\vdots

\end{align}

The ensuing generalized gradient descent on free energy can then be expressed compactly in terms of prediction errors, where (omitting high order terms):

: $\backslash begin\{align\}$

\dot{\tilde{\mu}}_u^{(i)} & =D\tilde{\mu}^{(u,i)}-\partial_u \tilde{\varepsilon}^{(i)}\cdot \Pi^{(i)}\tilde{\varepsilon}^{(i)}

-\Pi^{(i+1)}\tilde{\varepsilon}_u^{(i+1)} \\

\dot{\tilde{\mu}}_x^{(i)} & =D\tilde{\mu}^{(x,i)}-\partial_x \tilde{\varepsilon}^{(i)}\cdot \Pi^{(i)}\tilde{\varepsilon}^{(i)} \\

\\

\tilde{\varepsilon}_u^{(i)} & =\tilde{\mu}_u^{(i-1)} -\tilde{g}^{(i)} \\

\tilde{\varepsilon}_x^{(i)} & =D\tilde{\mu}_x^{(i)} -\tilde{f}^{(i)}

\end{align}

Here, $\backslash Pi^\{(i)\}$ is the precision of random fluctuations at the i-th level. This is known as generalized predictive coding [11], with linear predictive coding as a special case.

Generalized filtering has been primarily applied to biological timeseries—in particular functional magnetic resonance imaging and electrophysiological data. This is usually in the context of dynamic causal modelling to make inferences about the underlying architectures of (neuronal) systems generating data.J Daunizeau, O David, and K E Stephan, "[http://www.fil.ion.ucl.ac.uk/~jdaunize/publications/dcm_review.pdf Dynamic causal modelling: a critical review of the biophysical and statistical foundations] {{Webarchive|url=https://web.archive.org/web/20121207150056/http://www.fil.ion.ucl.ac.uk/~jdaunize/publications/dcm_review.pdf |date=2012-12-07 }}," Neuroimage, vol. 58, no. 2, pp. 312-22, 2011 It is also used to simulate inference in terms of generalized (hierarchical) predictive coding in the brain.K Friston, "[http://www.fil.ion.ucl.ac.uk/~karl/Hierarchical%20Models%20in%20the%20Brain.pdf Hierarchical models in the brain]," PLoS Comput. Biol., vol. 4, no. 11, p. e1000211, 2008.

• Dynamic Bayesian network
• Kalman filter
• Linear predictive coding
• Optimal control
• Particle filter
• Recursive Bayesian estimation
• System identification
• Variational Bayesian methods

{{Reflist|3}}

• [http://www.fil.ion.ucl.ac.uk/spm/software/ software] demonstrations and applications are available as academic freeware (as Matlab code) in the DEM toolbox of SPM
• [http://www.fil.ion.ucl.ac.uk/~karl/#_Bayesian_and_Variational papers] collection of technical and application papers

Category:Bayesian estimation

Category:Systems theory

Category:Control theory

Category:Nonlinear filters

Category:Linear filters

Category:Signal estimation

Category:Stochastic differential equations

Category:Markov models