Free energy principle#Free energy minimisation

In biophysics and cognitive science, the free energy principle is a mathematical principle describing a formal account of the representational capacities of physical systems: that is, why things that exist look as if they track properties of the systems to which they are coupled.{{cite journal

| last1 = Friston

| first1 = Karl

| title = The free-energy principle: a unified brain theory?

| journal = Nature Reviews Neuroscience

| volume = 11

| issue = 2

| pages = 127–138

| year = 2010

| doi = 10.1038/nrn2787

| pmid =20068583

| s2cid = 5053247

| url = https://www.nature.com/articles/nrn2787

| access-date =July 9, 2023

}} It establishes that the dynamics of physical systems minimise a quantity known as surprisal (which is the negative log probability of some outcome); or equivalently, its variational upper bound, called free energy. The principle is used especially in Bayesian approaches to brain function, but also some approaches to artificial intelligence; it is formally related to variational Bayesian methods and was originally introduced by Karl Friston as an explanation for embodied perception-action loops in neuroscience.{{cite journal | last1=Friston | first1=Karl | last2=Kilner | first2=James | last3=Harrison | first3=Lee | title=A free energy principle for the brain | journal=Journal of Physiology-Paris | volume=100 | issue=1–3 | year=2006 | doi=10.1016/j.jphysparis.2006.10.001 | pmid=17097864 | pages=70–87| s2cid=637885 |url=http://www.fil.ion.ucl.ac.uk/~karl/A%20free%20energy%20principle%20for%20the%20brain.pdf}}

The free energy principle models the behaviour of systems that are distinct from, but coupled to, another system (e.g., an embedding environment), where the degrees of freedom that implement the interface between the two systems is known as a Markov blanket. More formally, the free energy principle says that if a system has a "particular partition" (i.e., into particles, with their Markov blankets), then subsets of that system will track the statistical structure of other subsets (which are known as internal and external states or paths of a system).

The free energy principle is based on the Bayesian idea of the brain as an “inference engine.” Under the free energy principle, systems pursue paths of least surprise, or equivalently, minimize the difference between predictions based on their model of the world and their sense and associated perception. This difference is quantified by variational free energy and is minimized by continuous correction of the world model of the system, or by making the world more like the predictions of the system. By actively changing the world to make it closer to the expected state, systems can also minimize the free energy of the system. Friston assumes this to be the principle of all biological reaction.Shaun Raviv: [https://www.wired.com/story/karl-friston-free-energy-principle-artificial-intelligence/ The Genius Neuroscientist Who Might Hold the Key to True AI]. In: Wired, 13. November 2018 Friston also believes his principle applies to mental disorders as well as to artificial intelligence. AI implementations based on the active inference principle have shown advantages over other methods.

The free energy principle is a mathematical principle of information physics: much like the principle of maximum entropy or the principle of least action, it is true on mathematical grounds. To attempt to falsify the free energy principle is a category mistake, akin to trying to falsify calculus by making empirical observations. (One cannot invalidate a mathematical theory in this way; instead, one would need to derive a formal contradiction from the theory.) In a 2018 interview, Friston explained what it entails for the free energy principle to not be subject to falsification:{{Cite journal|last=Friston|first=Karl|date=2018|title=Of woodlice and men: A Bayesian account of cognition, life and consciousness. An interview with Karl Friston (by Martin Fortier & Daniel Friedman)|url=https://www.aliusresearch.org/bulletin02.html|journal=ALIUS Bulletin|volume=2|pages=17–43}}

I think it is useful to make a fundamental distinction at this point—that we can appeal to later.

The distinction is between a state and process theory; i.e., the difference between a normative principle that things may or may not conform to, and a process theory or hypothesis about how that principle is realized. Under this distinction, the free energy principle stands in stark distinction to things like predictive coding and the Bayesian brain hypothesis. This is because the free energy principle is what it is — a principle. Like Hamilton's principle of stationary action, it cannot be falsified. It cannot be disproven. In fact, there's not much you can do with it, unless you ask whether measurable systems conform to the principle. On the other hand, hypotheses that the brain performs some form of Bayesian inference or predictive coding are what they are—hypotheses. These hypotheses may or may not be supported by empirical evidence.

There are many examples of these hypotheses being supported by empirical evidence.{{Cite book|last=Friston|first=Karl|date=2022|title=Active Inference: The Free Energy Principle in Mind, Brain, and Behavior|publisher=MIT Press |isbn=9780262045353 |url=https://books.google.com/books?id=KXZ_zgEACAAJ&q=Table+9.1}}

The notion that self-organising biological systems – like a cell or brain – can be understood as minimising variational free energy is based upon Helmholtz's work on unconscious inferenceHelmholtz, H. (1866/1962). Concerning the perceptions in general. In Treatise on physiological optics (J. Southall, Trans., 3rd ed., Vol. III). New York: Dover. Available at https://web.archive.org/web/20180320133752/http://poseidon.sunyopt.edu/BackusLab/Helmholtz/ and subsequent treatments in psychology{{cite journal | title=Perceptions as hypotheses | journal=Philosophical Transactions of the Royal Society of London. B, Biological Sciences | volume=290 | issue=1038 | date=1980-07-08 | doi=10.1098/rstb.1980.0090 | pmid=6106237 | bibcode=1980RSPTB.290..181G | pages=181–197|jstor=2395424| last1=Gregory | first1=R. L. | doi-access= }} and machine learning.{{cite journal | last1=Dayan | first1=Peter | last2=Hinton | first2=Geoffrey E. | last3=Neal | first3=Radford M. | last4=Zemel | first4=Richard S. | title=The Helmholtz Machine | journal=Neural Computation | volume=7 | issue=5 | year=1995 | doi=10.1162/neco.1995.7.5.889 | pmid=7584891 | pages=889–904| s2cid=1890561 |url=http://www.gatsby.ucl.ac.uk/~dayan/papers/hm95.pdf| hdl=21.11116/0000-0002-D6D3-E | hdl-access=free }} Variational free energy is a function of observations and a probability density over their hidden causes. This variational density is defined in relation to a probabilistic model that generates predicted observations from hypothesized causes. In this setting, free energy provides an approximation to Bayesian model evidence.Beal, M. J. (2003). [http://www.cse.buffalo.edu/faculty/mbeal/papers/beal03.pdf Variational Algorithms for Approximate Bayesian Inference]. Ph.D. Thesis, University College London. Therefore, its minimisation can be seen as a Bayesian inference process. When a system actively makes observations to minimise free energy, it implicitly performs active inference and maximises the evidence for its model of the world.

However, free energy is also an upper bound on the self-information of outcomes, where the long-term average of surprise is entropy. This means that if a system acts to minimise free energy, it will implicitly place an upper bound on the entropy of the outcomes – or sensory states – it samples.{{cite arXiv | last1=Sakthivadivel | first1=Dalton | title=Towards a Geometry and Analysis for Bayesian Mechanics | year=2022 | class=math-ph | eprint=2204.11900}}{{cite journal | last1=Ramstead | first1=Maxwell | last2=Sakthivadivel | first2=Dalton | last3=Heins | first3=Conor | last4 = Koudahl | first4 = Magnus | last5 = Millidge | first5 = Beren | last6 = Da Costa | first6 = Lancelot | last7 = Klein | first7 = Brennan | last8 = Friston | first8 = Karl | title=On Bayesian mechanics: A physics of and by beliefs | journal=Interface Focus | year=2023 | volume=13 | issue=3 | doi=10.1098/rsfs.2022.0029 | pmid=37213925 | pmc=10198254 | arxiv=2205.11543| s2cid=249017997 }}

Active inference is closely related to the good regulator theorem{{cite journal | doi=10.1080/00207727008920220 | title=Every good regulator of a system must be a model of that system | year=1970 | last1=Conant | first1=Roger C. | last2=Ross Ashby | first2=W. | journal=International Journal of Systems Science | volume=1 | issue=2 | pages=89–97 }} and related accounts of self-organisation,Kauffman, S. (1993). [https://books.google.com/books?id=lZcSpRJz0dgC&dq=%22The+Origins+of+Order%3A+Self-Organization+and+Selection+in+Evolution%22&pg=PR13 The Origins of Order: Self-Organization and Selection in Evolution]. Oxford: Oxford University Press.Nicolis, G., & Prigogine, I. (1977). Self-organization in non-equilibrium systems. New York: John Wiley. such as self-assembly, pattern formation, autopoiesisMaturana, H. R., & Varela, F. (1980). [http://topologicalmedialab.net/xinwei/classes/readings/Maturana/autopoesis_and_cognition.pdf Autopoiesis: the organization of the living]. In V. F. Maturana HR (Ed.), Autopoiesis and Cognition. Dordrecht, Netherlands: Reidel. and practopoiesis.{{cite journal | doi=10.1016/j.jtbi.2015.03.003 | title=Practopoiesis: Or how life fosters a mind | year=2015 | last1=Nikolić | first1=Danko | journal=Journal of Theoretical Biology | volume=373 | pages=40–61 | pmid=25791287 | arxiv=1402.5332 | bibcode=2015JThBi.373...40N | s2cid=12680941 }} It addresses the themes considered in cybernetics, synergeticsHaken, H. (1983). Synergetics: An introduction. Non-equilibrium phase transition and self-organisation in physics, chemistry and biology (3rd ed.). Berlin: Springer Verlag. and embodied cognition. Because free energy can be expressed as the expected energy of observations under the variational density minus its entropy, it is also related to the maximum entropy principle.{{cite journal |doi=10.1103/PhysRev.106.620 |url=http://bayes.wustl.edu/etj/articles/theory.1.pdf|title=Information Theory and Statistical Mechanics |year=1957 |last1=Jaynes |first1=E. T. |journal=Physical Review |volume=106 |issue=4 |pages=620–630 |bibcode=1957PhRv..106..620J |s2cid=17870175 }} Finally, because the time average of energy is action, the principle of minimum variational free energy is a principle of least action. Active inference allowing for scale invariance has also been applied to other theories and domains. For instance, it has been applied to sociology,{{Cite journal |last1=Veissière |first1=Samuel P. L. |last2=Constant |first2=Axel |last3=Ramstead |first3=Maxwell J. D. |last4=Friston |first4=Karl J. |last5=Kirmayer |first5=Laurence J. |date=2020 |title=Thinking through other minds: A variational approach to cognition and culture |url=https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/thinking-through-other-minds-a-variational-approach-to-cognition-and-culture/9A10399BA85F428D5943DD847092C14A |journal=Behavioral and Brain Sciences |language=en |volume=43 |pages=e90 |doi=10.1017/S0140525X19001213 |pmid=31142395 |s2cid=169038428 |issn=0140-525X}}{{Cite journal |last1=Ramstead |first1=Maxwell J. D. |last2=Constant |first2=Axel |last3=Badcock |first3=Paul B. |last4=Friston |first4=Karl J. |date=2019-12-01 |title=Variational ecology and the physics of sentient systems |journal=Physics of Life Reviews |series=Physics of Mind |language=en |volume=31 |pages=188–205 |doi=10.1016/j.plrev.2018.12.002 |pmid=30655223 |pmc=6941227 |bibcode=2019PhLRv..31..188R |issn=1571-0645}}{{Cite journal |last1=Albarracin |first1=Mahault |last2=Demekas |first2=Daphne |last3=Ramstead |first3=Maxwell J. D. |last4=Heins |first4=Conor |date=April 2022 |title=Epistemic Communities under Active Inference |journal=Entropy |language=en |volume=24 |issue=4 |pages=476 |doi=10.3390/e24040476 |pmid=35455140 |pmc=9027706 |bibcode=2022Entrp..24..476A |issn=1099-4300 |doi-access=free }}{{Cite journal |last1=Albarracin |first1=Mahault |last2=Constant |first2=Axel |last3=Friston |first3=Karl J. |last4=Ramstead |first4=Maxwell James D. |date=2021 |title=A Variational Approach to Scripts |journal=Frontiers in Psychology |volume=12 |page=585493 |doi=10.3389/fpsyg.2021.585493 |pmid=34354621 |pmc=8329037 |issn=1664-1078 |doi-access=free }} linguistics and communication,{{Cite journal |last1=Friston |first1=Karl J. |last2=Parr |first2=Thomas |last3=Yufik |first3=Yan |last4=Sajid |first4=Noor |last5=Price |first5=Catherine J. |last6=Holmes |first6=Emma |date=2020-11-01 |title=Generative models, linguistic communication and active inference |journal=Neuroscience & Biobehavioral Reviews |language=en |volume=118 |pages=42–64 |doi=10.1016/j.neubiorev.2020.07.005 |pmid=32687883 |pmc=7758713 |issn=0149-7634}}{{Cite journal |last1=Tison |first1=Remi |last2=Poirier |first2=Pierre |date=2021-10-02 |title=Communication as Socially Extended Active Inference: An Ecological Approach to Communicative Behavior |url=https://doi.org/10.1080/10407413.2021.1965480 |journal=Ecological Psychology |volume=33 |issue=3–4 |pages=197–235 |doi=10.1080/10407413.2021.1965480 |s2cid=238703201 |issn=1040-7413}}{{Cite journal |last1=Friston |first1=Karl J. |last2=Frith |first2=Christopher D. |date=2015-07-01 |title=Active inference, communication and hermeneutics |journal=Cortex |series=Special issue: Prediction in speech and language processing |language=en |volume=68 |pages=129–143 |doi=10.1016/j.cortex.2015.03.025 |pmid=25957007 |pmc=4502445 |issn=0010-9452}} semiotics,{{Cite journal |last=Kerusauskaite |first=Skaiste |date=2023-06-01 |title=Role of Culture in Meaning Making: Bridging Semiotic Cultural Psychology and Active Inference |url=https://doi.org/10.1007/s12124-022-09744-x |journal=Integrative Psychological and Behavioral Science |language=en |volume=57 |issue=2 |pages=432–443 |doi=10.1007/s12124-022-09744-x |pmid=36585542 |s2cid=255366405 |issn=1936-3567}}{{Cite book |last1=García |first1=Adolfo M. |url=https://books.google.com/books?id=hPCKEAAAQBAJ&dq=active+inference+semiotics&pg=PT90 |title=The Routledge Handbook of Semiosis and the Brain |last2=Ibáñez |first2=Agustín |date=2022-11-14 |publisher=Taylor & Francis |isbn=978-1-000-72877-4 |language=en}} and epidemiology {{Cite journal |last1=Bottemanne |first1=Hugo |last2=Friston |first2=Karl J. |date=2021-12-01 |title=An active inference account of protective behaviours during the COVID-19 pandemic |url=https://doi.org/10.3758/s13415-021-00947-0 |journal=Cognitive, Affective, & Behavioral Neuroscience |language=en |volume=21 |issue=6 |pages=1117–1129 |doi=10.3758/s13415-021-00947-0 |issn=1531-135X |pmc=8518276 |pmid=34652601}} among others.

Negative free energy is formally equivalent to the evidence lower bound, which is commonly used in machine learning to train generative models, such as variational autoencoders.

{{citations needed|date=March 2025}}

Image:MarokovBlanketFreeEnergyFigure.jpg

Active inference applies the techniques of approximate Bayesian inference to infer the causes of sensory data from a 'generative' model of how that data is caused and then uses these inferences to guide action.

Bayes' rule characterizes the probabilistically optimal inversion of such a causal model, but applying it is typically computationally intractable, leading to the use of approximate methods.

In active inference, the leading class of such approximate methods are variational methods, for both practical and theoretical reasons: practical, as they often lead to simple inference procedures; and theoretical, because they are related to fundamental physical principles, as discussed above.

These variational methods proceed by minimizing an upper bound on the divergence between the Bayes-optimal inference (or 'posterior') and its approximation according to the method.

This upper bound is known as the free energy, and we can accordingly characterize perception as the minimization of the free energy with respect to inbound sensory information, and action as the minimization of the same free energy with respect to outbound action information.

This holistic dual optimization is characteristic of active inference, and the free energy principle is the hypothesis that all systems which perceive and act can be characterized in this way.

In order to exemplify the mechanics of active inference via the free energy principle, a generative model must be specified, and this typically involves a collection of probability density functions which together characterize the causal model.

One such specification is as follows.

The system is modelled as inhabiting a state space $X$, in the sense that its states form the points of this space.

The state space is then factorized according to $X\; =\; \backslash Psi\backslash times\; S\backslash times\; A\backslash times\; R$, where $\backslash Psi$ is the space of 'external' states that are 'hidden' from the agent (in the sense of not being directly perceived or accessible), $S$ is the space of sensory states that are directly perceived by the agent, $A$ is the space of the agent's possible actions, and $R$ is a space of 'internal' states that are private to the agent.

Keeping with the Figure 1, note that in the following the $\backslash dot\{\backslash psi\},\; \backslash psi,\; s,\; a$ and $\backslash mu$ are functions of (continuous) time $t$. The generative model is the specification of the following density functions:

• A sensory model, $p\_S:S\; \backslash times\; \backslash Psi\backslash times\; A\; \backslash to\; \backslash mathbb\{R\}$, often written as $p\_S(s\; \backslash mid\; \backslash psi,\; a)$, characterizing the likelihood of sensory data given external states and actions;
• a stochastic model of the environmental dynamics, $p\_\backslash Psi:\; \backslash Psi\; \backslash times\; \backslash Psi\; \backslash times\; A\; \backslash to\; \backslash mathbb\{R\}$, often written $p\_\backslash Psi(\backslash dot\{\backslash psi\}\; \backslash mid\; \backslash psi,\; a)$, characterizing how the external states are expected by the agent to evolve over time $t$, given the agent's actions;
• an action model, $p\_A:\; A\; \backslash times\; R\; \backslash times\; S\; \backslash to\; \backslash mathbb\{R\}$, written $p\_A(a\; \backslash mid\; \backslash mu,\; s)$, characterizing how the agent's actions depend upon its internal states and sensory data; and
• an internal model, $p\_R:\; R\; \backslash times\; S\; \backslash to\; \backslash mathbb\{R\}$, written $p\_R(\backslash mu\; \backslash mid\; s)$, characterizing how the agent's internal states depend upon its sensory data.

These density functions determine the factors of a "joint model", which represents the complete specification of the generative model, and which can be written as

: $p(\backslash dot\{\backslash psi\},\; s,\; a,\; \backslash mu\; \backslash mid\; \backslash psi)\; =\; p\_S(s\; \backslash mid\; \backslash psi,\; a)p\_\backslash Psi(\backslash dot\{\backslash psi\}\; \backslash mid\; \backslash psi,\; a)p\_A(a\; \backslash mid\; \backslash mu,\; s)p\_R(\backslash mu\; \backslash mid\; s)$.

Bayes' rule then determines the "posterior density" $p\; \_\{\backslash text\{Bayes\}\}(\backslash dot\{\backslash psi\}\; |\; s,\; a,\; \backslash mu,\; \backslash psi)$, which expresses a probabilistically optimal belief about the external state $\backslash dot\{\backslash psi\}$ given the preceding state $\backslash psi$ and the agent's actions, sensory signals, and internal states.

Since computing $p\_\{\backslash text\{Bayes\}\}$ is computationally intractable, the free energy principle asserts the existence of a "variational density" $q(\backslash dot\{\backslash psi\}\; |\; s,\; a,\; \backslash mu,\; \backslash psi)$, where $q$ is an approximation to $p\_\{\backslash text\{Bayes\}\}$.

One then defines the free energy as

: $\backslash begin\{align\}$

\underset{\mathrm{free-energy}} {\underbrace{F(\mu, a\, ; s)}} &= \underset{\text{expected energy}} {\underbrace{ \mathbb{E}_{q(\dot{\psi})}[-\log p(\dot{\psi}, s, a, \mu \mid \psi)]}} - \underset{\mathrm{entropy}} {\underbrace{ \mathbb{H}[q(\dot{\psi} \mid s, a, \mu, \psi)]}}\\

&= \underset{\mathrm{surprise}} {\underbrace{ -\log p(s)}} + \underset{\mathrm{divergence}} {\underbrace{ \mathbb{KL}[q(\dot{\psi} \mid s, a, \mu, \psi) \parallel p_{\text{Bayes}}(\dot{\psi} \mid s, a, \mu, \psi)]}} \\

&\geq \underset{\mathrm{surprise}} {\underbrace{ -\log p(s)}}

\end{align}

and defines action and perception as the joint optimization problem

: $\backslash begin\{align\}$

\mu^* &= \underset{\mu}{\operatorname{arg\,min}} \{ F(\mu, a \,;\, s)) \} \\

a^* &= \underset{a}{\operatorname{arg\,min}} \{ F(\mu^*, a \,;\, s) \}

\end{align}

where the internal states $\backslash mu$ are typically taken to encode the parameters of the 'variational' density $q$ and hence the agent's "best guess" about the posterior belief over $\backslash Psi$.

Note that the free energy is also an upper bound on a measure of the agent's (marginal, or average) sensory surprise, and hence free energy minimization is often motivated by the minimization of surprise.

Free energy minimisation has been proposed as a hallmark of self-organising systems when cast as random dynamical systems.{{cite journal |doi=10.1007/BF01193705 |url=https://www.researchgate.net/publication/227072665|title=Attractors for random dynamical systems |year=1994 |last1=Crauel |first1=Hans |last2=Flandoli |first2=Franco |journal=Probability Theory and Related Fields |volume=100 |issue=3 |pages=365–393 |s2cid=122609512 |doi-access=free }} This formulation rests on a Markov blanket (comprising action and sensory states) that separates internal and external states. If internal states and action minimise free energy, then they place an upper bound on the entropy of sensory states:

: $\backslash lim\_\{T\backslash to\backslash infty\}\; \backslash frac\{1\}\{T\}\; \backslash underset\{\backslash text\{free-action\}\}\; \{\backslash underbrace\{\backslash int\_0^T\; F(s(t),\backslash mu\; (t))\backslash ,dt\}\}\; \backslash ge$

\lim_{T\to\infty} \frac{1}{T} \int_0^T \underset{\text{surprise}}{\underbrace{-\log p(s(t)\mid m)}} \, dt = H[p(s\mid m)]

This is because – under ergodic assumptions – the long-term average of surprise is entropy. This bound resists a natural tendency to disorder – of the sort associated with the second law of thermodynamics and the fluctuation theorem. However, formulating a unifying principle for the life sciences in terms of concepts from statistical physics, such as random dynamical system, non-equilibrium steady state and ergodicity, places substantial constraints on the theoretical and empirical study of biological systems with the risk of obscuring all features that make biological systems interesting kinds of self-organizing systems.{{cite journal | doi=10.1007/s10539-021-09818-x | title=Non-equilibrium thermodynamics and the free energy principle in biology | year=2021 | last1=Colombo | first1=Matteo | last2=Palacios | first2=Patricia | journal=Biology & Philosophy | volume=36 | issue=5 | s2cid=235803361 | doi-access=free }}

All Bayesian inference can be cast in terms of free energy minimisation{{cite journal |url=http://authors.library.caltech.edu/13697/1/ROWnc99.pdf |doi=10.1162/089976699300016674|title=A Unifying Review of Linear Gaussian Models |year=1999 |last1=Roweis |first1=Sam |last2=Ghahramani |first2=Zoubin |journal=Neural Computation |volume=11 |issue=2 |pages=305–345 |pmid=9950734 |s2cid=2590898 }}{{Failed verification|date=April 2020}}. When free energy is minimised with respect to internal states, the Kullback–Leibler divergence between the variational and posterior density over hidden states is minimised. This corresponds to approximate Bayesian inference – when the form of the variational density is fixed – and exact Bayesian inference otherwise. Free energy minimisation therefore provides a generic description of Bayesian inference and filtering (e.g., Kalman filtering). It is also used in Bayesian model selection, where free energy can be usefully decomposed into complexity and accuracy:

: $\backslash underset\{\backslash text\{free-energy\}\}\; \{\backslash underbrace\{\; F(s,\backslash mu)\}\}\; =\; \backslash underset\{\backslash text\{complexity\}\}\; \{\backslash underbrace\{\; D\_\backslash mathrm\{KL\}[q(\backslash psi\backslash mid\backslash mu)\backslash parallel\; p(\backslash psi\backslash mid\; m)]\}\}\; -\; \backslash underset\{\backslash mathrm\{accuracy\}\}\; \{\backslash underbrace\{E\_q[\backslash log\; p(s\backslash mid\backslash psi,m)]\}\}$

Models with minimum free energy provide an accurate explanation of data, under complexity costs; cf. Occam's razor and more formal treatments of computational costs.{{cite journal | url=http://rspa.royalsocietypublishing.org/content/469/2153/20120683 | doi=10.1098/rspa.2012.0683 | title=Thermodynamics as a theory of decision-making with information-processing costs | year=2013 | last1=Ortega | first1=Pedro A. | last2=Braun | first2=Daniel A. | journal=Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences | volume=469 | issue=2153 | arxiv=1204.6481 | bibcode=2013RSPSA.46920683O | s2cid=28080508 }} Here, complexity is the divergence between the variational density and prior beliefs about hidden states (i.e., the effective degrees of freedom used to explain the data).

Variational free energy is an information-theoretic functional and is distinct from thermodynamic (Helmholtz) free energy.{{cite journal |url=http://rscweb.anu.edu.au/~evans/papers/NEFET.pdf |doi=10.1080/0026897031000085173|title=A non-equilibrium free energy theorem for deterministic systems |year=2003 |last1=Evans |first1=Denis J. |journal=Molecular Physics |volume=101 |issue=10 |pages=1551–1554 |bibcode=2003MolPh.101.1551E |s2cid=15129000 }} However, the complexity term of variational free energy shares the same fixed point as Helmholtz free energy (under the assumption the system is thermodynamically closed but not isolated). This is because if sensory perturbations are suspended (for a suitably long period of time), complexity is minimised (because accuracy can be neglected). At this point, the system is at equilibrium and internal states minimise Helmholtz free energy, by the principle of minimum energy.{{cite journal | arxiv=cond-mat/9610209 | doi=10.1103/PhysRevLett.78.2690 | title=Nonequilibrium Equality for Free Energy Differences | year=1997 | last1=Jarzynski | first1=C. | journal=Physical Review Letters | volume=78 | issue=14 | pages=2690–2693 | bibcode=1997PhRvL..78.2690J | s2cid=16112025 }}

Free energy minimisation is equivalent to maximising the mutual information between sensory states and internal states that parameterise the variational density (for a fixed entropy variational density). This relates free energy minimization to the principle of minimum redundancy.{{cite arXiv | last1=Sakthivadivel | first1=Dalton | title=Towards a Geometry and Analysis for Bayesian Mechanics | year=2022 | class=math-ph |eprint=2204.11900}}

Free energy minimisation provides a useful way to formulate normative (Bayes optimal) models of neuronal inference and learning under uncertainty{{cite journal |url=http://www.fil.ion.ucl.ac.uk/~karl/The%20free-energy%20principle%20A%20unified%20brain%20theory.pdf |doi=10.1038/nrn2787|title=The free-energy principle: A unified brain theory? |year=2010 |last1=Friston |first1=Karl |journal=Nature Reviews Neuroscience |volume=11 |issue=2 |pages=127–138 |pmid=20068583 |s2cid=5053247 }} and therefore subscribes to the Bayesian brain hypothesis.{{cite journal |doi=10.1016/j.tins.2004.10.007 |url=http://mrl.isr.uc.pt/pub/bscw.cgi/d27540/ReviewKnillPouget2.pdf |title=The Bayesian brain: The role of uncertainty in neural coding and computation |year=2004 |last1=Knill |first1=David C. |last2=Pouget |first2=Alexandre |journal=Trends in Neurosciences |volume=27 |issue=12 |pages=712–719 |pmid=15541511 |s2cid=9870936 |access-date=2013-05-31 |archive-date=2016-03-04 |archive-url=https://web.archive.org/web/20160304044221/http://mrl.isr.uc.pt/pub/bscw.cgi/d27540/ReviewKnillPouget2.pdf |url-status=dead }} The neuronal processes described by free energy minimisation depend on the nature of hidden states: $\backslash Psi\; =\; X\; \backslash times\; \backslash Theta\; \backslash times\; \backslash Pi$ that can comprise time-dependent variables, time-invariant parameters and the precision (inverse variance or temperature) of random fluctuations. Minimising variables, parameters, and precision correspond to inference, learning, and the encoding of uncertainty, respectively.

Free energy minimisation formalises the notion of unconscious inference in perception and provides a normative (Bayesian) theory of neuronal processing. The associated process theory of neuronal dynamics is based on minimising free energy through gradient descent. This corresponds to generalised Bayesian filtering (where ~ denotes a variable in generalised coordinates of motion and $D$ is a derivative matrix operator):{{cite journal | doi=10.1155/2010/621670 | doi-access=free | title=Generalised Filtering | year=2010 | last1=Friston | first1=Karl | last2=Stephan | first2=Klaas | last3=Li | first3=Baojuan | last4=Daunizeau | first4=Jean | journal=Mathematical Problems in Engineering | volume=2010 | pages=1–34 }}

: $\backslash dot\{\backslash tilde\{\backslash mu\}\}\; =\; D\; \backslash tilde\{\backslash mu\}\; -\; \backslash partial\_\{\backslash mu\}F(s,\backslash mu)\backslash Big|\_\{\backslash mu\; =\; \backslash tilde\{\backslash mu\}\}$

Usually, the generative models that define free energy are non-linear and hierarchical (like cortical hierarchies in the brain). Special cases of generalised filtering include Kalman filtering, which is formally equivalent to predictive coding{{cite journal |doi=10.1016/j.tins.2004.10.007 |url=https://www.cs.utexas.edu/users/dana/nn.pdf|title=The Bayesian brain: The role of uncertainty in neural coding and computation |year=2004 |last1=Knill |first1=David C. |last2=Pouget |first2=Alexandre |journal=Trends in Neurosciences |volume=27 |issue=12 |pages=712–719 |pmid=15541511 |s2cid=9870936 }} – a popular metaphor for message passing in the brain. Under hierarchical models, predictive coding involves the recurrent exchange of ascending (bottom-up) prediction errors and descending (top-down) predictions{{cite journal |doi=10.1007/BF00198477 |url=http://cs.brown.edu/people/tld/projects/cortex/course/suggested_reading_list/supplements/documents/MumfordBC-92.pdf|title=On the computational architecture of the neocortex |year=1992 |last1=Mumford |first1=D. |journal=Biological Cybernetics |volume=66 |issue=3 |pages=241–251 |pmid=1540675 |s2cid=14303625 }} that is consistent with the anatomy and physiology of sensory{{cite journal | doi=10.1016/j.neuron.2012.10.038 | title=Canonical Microcircuits for Predictive Coding | year=2012 | last1=Bastos | first1=Andre M. | last2=Usrey | first2=W. Martin | last3=Adams | first3=Rick A. | last4=Mangun | first4=George R. | last5=Fries | first5=Pascal | last6=Friston | first6=Karl J. | journal=Neuron | volume=76 | issue=4 | pages=695–711 | pmid=23177956 | pmc=3777738 }} and motor systems.{{cite journal | doi=10.1007/s00429-012-0475-5 | title=Predictions not commands: Active inference in the motor system | year=2013 | last1=Adams | first1=Rick A. | last2=Shipp | first2=Stewart | last3=Friston | first3=Karl J. | journal=Brain Structure and Function | volume=218 | issue=3 | pages=611–643 | pmid=23129312 | pmc=3637647 }}

In predictive coding, optimising model parameters through a gradient descent on the time integral of free energy (free action) reduces to associative or Hebbian plasticity and is associated with synaptic plasticity in the brain.

Optimizing the precision parameters corresponds to optimizing the gain of prediction errors (cf., Kalman gain). In neuronally plausible implementations of predictive coding, this corresponds to optimizing the excitability of superficial pyramidal cells and has been interpreted in terms of attentional gain.{{cite journal | doi=10.3389/fnhum.2010.00215 | doi-access=free | title=Attention, Uncertainty, and Free-Energy | year=2010 | last1=Friston | first1=Karl J. | last2=Feldman | first2=Harriet | journal=Frontiers in Human Neuroscience | volume=4 | page=215 | pmid=21160551 | pmc=3001758 }}

File:PESAIM.jpg

With regard to the top-down vs. bottom-up controversy, which has been addressed as a major open problem of attention, a computational model has succeeded in illustrating the circular nature of the interplay between top-down and bottom-up mechanisms. Using an established emergent model of attention, namely SAIM, the authors proposed a model called PE-SAIM, which, in contrast to the standard version, approaches selective attention from a top-down position. The model takes into account the transmission of prediction errors to the same level or a level above, in order to minimise the energy function that indicates the difference between the data and its cause, or, in other words, between the generative model and the posterior. To increase validity, they also incorporated neural competition between stimuli into their model. A notable feature of this model is the reformulation of the free energy function only in terms of prediction errors during task performance:

$\backslash dfrac\{\backslash partial\; E^\{total\}(Y^\{VP\},X^\{SN\},x^\{CN\},y^\{KN\})\}\{\backslash partial\; y^\{SN\}\_\{mn\}\}=x^\{CN\}\_\{mn\}-b^\{CN\}\backslash varepsilon^\{CN\}\_\{nm\}+b^\{CN\}\backslash sum\_\{k\}(\backslash varepsilon^\{KN\}\_\{knm\})$

where $E^\{total\}$ is the total energy function of the neural networks entail, and $\backslash varepsilon^\{KN\}\_\{knm\}$ is the prediction error between the generative model (prior) and posterior changing over time.{{cite journal | doi=10.1098/rsif.2018.0344 | title=Excitatory versus inhibitory feedback in Bayesian formulations of scene construction | year=2019 | last1=Abadi | first1=Alireza Khatoon | last2=Yahya | first2=Keyvan | last3=Amini | first3=Massoud | last4=Friston | first4=Karl | last5=Heinke | first5=Dietmar | journal=Journal of the Royal Society Interface | volume=16 | issue=154 | pmid=31039693 | pmc=6544897 }}

Comparing the two models reveals a notable similarity between their respective results while also highlighting a remarkable discrepancy, whereby – in the standard version of the SAIM – the model's focus is mainly upon the excitatory connections, whereas in the PE-SAIM, the inhibitory connections are leveraged to make an inference. The model has also proved to be fit to predict the EEG and fMRI data drawn from human experiments with high precision. In the same vein, Yahya et al. also applied the free energy principle to propose a computational model for template matching in covert selective visual attention that mostly relies on SAIM.{{cite journal | doi=10.1007/s10339-013-0597-6 | title=12th Biannual Conference of the German Cognitive Science Society (KogWis 2014) | journal=Cognitive Processing | year=2014 | volume=15 | page=107 | s2cid=10121398 | doi-access=free }} According to this study, the total free energy of the whole state-space is reached by inserting top-down signals in the original neural networks, whereby we derive a dynamical system comprising both feed-forward and backward prediction error.

When gradient descent is applied to action $\backslash dot\{a\}\; =\; -\backslash partial\_aF(s,\backslash tilde\{\backslash mu\})$, motor control can be understood in terms of classical reflex arcs that are engaged by descending (corticospinal) predictions. This provides a formalism that generalizes the equilibrium point solution – to the degrees of freedom problem{{cite journal |doi=10.1017/S0140525X0004070X |url=http://e.guigon.free.fr/rsc/article/FeldmanLevin95.pdf |title=The origin and use of positional frames of reference in motor control |year=1995 |last1=Feldman |first1=Anatol G. |last2=Levin |first2=Mindy F. |journal=Behavioral and Brain Sciences |volume=18 |issue=4 |pages=723–744 |s2cid=145164477 |access-date=2013-05-31 |archive-date=2014-03-29 |archive-url=https://web.archive.org/web/20140329220749/http://e.guigon.free.fr/rsc/article/FeldmanLevin95.pdf |url-status=dead }} – to movement trajectories.

Active inference is related to optimal control by replacing value or cost-to-go functions with prior beliefs about state transitions or flow.{{cite journal |doi=10.1016/j.neuron.2011.10.018 |url=http://www.fil.ion.ucl.ac.uk/~karl/What%20Is%20Optimal%20about%20Motor%20Control.pdf|title=What is Optimal about Motor Control? |year=2011 |last1=Friston |first1=Karl |journal=Neuron |volume=72 |issue=3 |pages=488–498 |pmid=22078508 |s2cid=13912462 }} This exploits the close connection between Bayesian filtering and the solution to the Bellman equation. However, active inference starts with (priors over) flow $f\; =\; \backslash Gamma\; \backslash cdot\; \backslash nabla\; V\; +\; \backslash nabla\; \backslash times\; W$ that are specified with scalar $V(x)$ and vector $W(x)$ value functions of state space (cf., the Helmholtz decomposition). Here, $\backslash Gamma$ is the amplitude of random fluctuations and cost is $c(x)\; =\; f\; \backslash cdot\; \backslash nabla\; V\; +\; \backslash nabla\; \backslash cdot\; \backslash Gamma\; \backslash cdot\; V$. The priors over flow $p(\backslash tilde\{x\}\backslash mid\; m)$ induce a prior over states $p(x\backslash mid\; m)\; =\; \backslash exp\; (V(x))$ that is the solution to the appropriate forward Kolmogorov equations.{{cite journal | doi=10.1155/2012/937860 | doi-access=free | title=Free Energy, Value, and Attractors | year=2012 | last1=Friston | first1=Karl | last2=Ao | first2=Ping | journal=Computational and Mathematical Methods in Medicine | volume=2012 | pages=1–27 | pmid=22229042 | pmc=3249597 }} In contrast, optimal control optimises the flow, given a cost function, under the assumption that $W\; =\; 0$ (i.e., the flow is curl free or has detailed balance). Usually, this entails solving backward Kolmogorov equations.{{cite journal | arxiv=physics/0505066 | doi=10.1088/1742-5468/2005/11/P11011 | title=Path integrals and symmetry breaking for optimal control theory | year=2005 | last1=Kappen | first1=H. J. | journal=Journal of Statistical Mechanics: Theory and Experiment | volume=2005 | issue=11 | pages=P11011 | bibcode=2005JSMTE..11..011K | s2cid=87027 }}

Optimal decision problems (usually formulated as partially observable Markov decision processes) are treated within active inference by absorbing utility functions into prior beliefs. In this setting, states that have a high utility (low cost) are states an agent expects to occupy. By equipping the generative model with hidden states that model control, policies (control sequences) that minimise variational free energy lead to high utility states.{{cite journal |doi=10.1007/s00422-012-0512-8 |doi-access=free|title=Active inference and agency: Optimal control without cost functions |year=2012 |last1=Friston |first1=Karl |last2=Samothrakis |first2=Spyridon |last3=Montague |first3=Read |journal=Biological Cybernetics |volume=106 |issue=8–9 |pages=523–541 |pmid=22864468 |hdl=10919/78836 |hdl-access=free }}

Neurobiologically, neuromodulators such as dopamine are considered to report the precision of prediction errors by modulating the gain of principal cells encoding prediction error.{{cite journal | doi=10.1371/journal.pcbi.1002327 | title=Dopamine, Affordance and Active Inference | year=2012 | last1=Friston | first1=Karl J. | last2=Shiner | first2=Tamara | last3=Fitzgerald | first3=Thomas | last4=Galea | first4=Joseph M. | last5=Adams | first5=Rick | last6=Brown | first6=Harriet | last7=Dolan | first7=Raymond J. | last8=Moran | first8=Rosalyn|author8-link=Rosalyn Moran | last9=Stephan | first9=Klaas Enno | last10=Bestmann | first10=Sven | journal=PLOS Computational Biology | volume=8 | issue=1 | pages=e1002327 | pmid=22241972 | pmc=3252266 | bibcode=2012PLSCB...8E2327F | doi-access=free }} This is closely related to – but formally distinct from – the role of dopamine in reporting prediction errors per se{{cite journal |doi=10.1126/science.1077349 |url=http://e.guigon.free.fr/rsc/article/FiorilloEtAl03.pdf |title=Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons |year=2003 |last1=Fiorillo |first1=Christopher D. |last2=Tobler |first2=Philippe N. |last3=Schultz |first3=Wolfram |journal=Science |volume=299 |issue=5614 |pages=1898–1902 |pmid=12649484 |bibcode=2003Sci...299.1898F |s2cid=2363255 |access-date=2013-05-31 |archive-date=2016-03-04 |archive-url=https://web.archive.org/web/20160304045504/http://e.guigon.free.fr/rsc/article/FiorilloEtAl03.pdf |url-status=dead }} and related computational accounts.{{cite journal |doi=10.1162/0898929052880093 |url=http://ski.cog.brown.edu/papers/Frank_JOCN.pdf|title=Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism |year=2005 |last1=Frank |first1=Michael J. |journal=Journal of Cognitive Neuroscience |volume=17 |issue=1 |pages=51–72 |pmid=15701239 |s2cid=7414727 }}

Active inference has been used to address a range of issues in cognitive neuroscience, brain function and neuropsychiatry, including action observation,{{cite journal |doi=10.1007/s00422-011-0424-z |url=http://www.fil.ion.ucl.ac.uk/~karl/Action%20understanding%20and%20active%20inference.pdf|title=Action understanding and active inference |year=2011 |last1=Friston |first1=Karl |last2=Mattout |first2=Jérémie |last3=Kilner |first3=James |journal=Biological Cybernetics |volume=104 |issue=1–2 |pages=137–160 |pmid=21327826 |pmc=3491875 }} mirror neurons,{{cite journal |doi=10.1007/s10339-007-0170-2 |url=http://www.fil.ion.ucl.ac.uk/~karl/Predictive%20coding%20an%20account%20of%20the%20mirror%20neuron%20system.pdf|title=Predictive coding: An account of the mirror neuron system |year=2007 |last1=Kilner |first1=James M. |last2=Friston |first2=Karl J. |last3=Frith |first3=Chris D. |journal=Cognitive Processing |volume=8 |issue=3 |pages=159–166 |pmid=17429704 |pmc=2649419 }} saccades and visual search,{{cite journal | doi=10.3389/fpsyg.2012.00151 | doi-access=free | title=Perceptions as Hypotheses: Saccades as Experiments | year=2012 | last1=Friston | first1=Karl | last2=Adams | first2=Rick A. | last3=Perrinet | first3=Laurent | last4=Breakspear | first4=Michael | journal=Frontiers in Psychology | volume=3 | page=151 | pmid=22654776 | pmc=3361132 }}{{cite journal | doi=10.1371/journal.pone.0190429 | doi-access=free | title=Human visual exploration reduces uncertainty about the sensed world | year=2018 | last1=Mirza | first1=M. Berk | last2=Adams | first2=Rick A. | last3=Mathys | first3=Christoph | last4=Friston | first4=Karl J. | journal=PLOS ONE | volume=13 | issue=1 | pages=e0190429 | pmid=29304087 | pmc=5755757 | bibcode=2018PLoSO..1390429M }} eye movements,{{cite journal | doi=10.1007/s00422-014-0620-8 | title=Active inference, eye movements and oculomotor delays | year=2014 | last1=Perrinet | first1=Laurent U. | last2=Adams | first2=Rick A. | last3=Friston | first3=Karl J. | journal=Biological Cybernetics | volume=108 | issue=6 | pages=777–801 | pmid=25128318 | pmc=4250571 }} sleep,{{cite journal |doi=10.1016/j.pneurobio.2012.05.003 |doi-access=free|title=Waking and dreaming consciousness: Neurobiological and functional considerations |year=2012 |last1=Hobson |first1=J.A. |last2=Friston |first2=K.J. |journal=Progress in Neurobiology |volume=98 |issue=1 |pages=82–98 |pmid=22609044 |pmc=3389346 }} illusions,{{cite journal | doi=10.3389/fpsyg.2012.00043 | doi-access=free | title=Free-Energy and Illusions: The Cornsweet Effect | year=2012 | last1=Brown | first1=Harriet | last2=Friston | first2=Karl J. | journal=Frontiers in Psychology | volume=3 | page=43 | pmid=22393327 | pmc=3289982 }} attention, action selection, consciousness,{{Cite journal|last1=Rudrauf|first1=David|last2=Bennequin|first2=Daniel|last3=Granic|first3=Isabela|last4=Landini|first4=Gregory|last5=Friston|first5=Karl|last6=Williford|first6=Kenneth|date=2017-09-07|title=A mathematical model of embodied consciousness|journal=Journal of Theoretical Biology|volume=428|pages=106–131|doi=10.1016/j.jtbi.2017.05.032|pmid=28554611|bibcode=2017JThBi.428..106R |url=http://discovery.ucl.ac.uk/10057795/1/DR_et_al_A_math_model_of_embodied_consciousness_JTBiol_final_revision_for_submission.pdf}}{{Cite journal|title=The Projective Consciousness Model and Phenomenal Selfhood|journal = Frontiers in Psychology|volume = 9|pages = 2571|last1=K|first1=Williford|last2=D|first2=Bennequin|date=2018-12-17|language=en|pmid=30618988|pmc = 6304424|last3=K|first3=Friston|last4=D|first4=Rudrauf|doi = 10.3389/fpsyg.2018.02571|doi-access = free}} hysteria{{cite journal |doi=10.1093/brain/aws129 |url=http://www.fil.ion.ucl.ac.uk/~karl/A%20Bayesian%20account%20of%20hysteria.pdf|title=A Bayesian account of 'hysteria' |year=2012 |last1=Edwards |first1=M. J. |last2=Adams |first2=R. A. |last3=Brown |first3=H. |last4=Parees |first4=I. |last5=Friston |first5=K. J. |journal=Brain |volume=135 |issue=11 |pages=3495–3512 |pmid=22641838 |pmc=3501967 }} and psychosis.{{cite journal | doi=10.1371/journal.pone.0047502 | doi-access=free | title=Smooth Pursuit and Visual Occlusion: Active Inference and Oculomotor Control in Schizophrenia | year=2012 | last1=Adams | first1=Rick A. | last2=Perrinet | first2=Laurent U. | last3=Friston | first3=Karl | journal=PLOS ONE | volume=7 | issue=10 | pages=e47502 | pmid=23110076 | pmc=3482214 | bibcode=2012PLoSO...747502A }} Explanations of action in active inference often depend on the idea that the brain has 'stubborn predictions' that it cannot update, leading to actions that cause these predictions to come true.{{Cite journal|last1=Yon|first1=Daniel|last2=Lange|first2=Floris P. de|last3=Press|first3=Clare|date=2019-01-01|title=The Predictive Brain as a Stubborn Scientist|url=https://www.cell.com/trends/cognitive-sciences/abstract/S1364-6613(18)30239-0|journal=Trends in Cognitive Sciences|language=en|volume=23|issue=1|pages=6–8|doi=10.1016/j.tics.2018.10.003|pmid=30429054|s2cid=53280000}}

• {{annotated link|Action-specific perception}}
• {{annotated link|Affordance}}
• {{annotated link|Autopoiesis}}
• {{annotated link|Bayesian approaches to brain function}}
• Constructal law - Law of design evolution in nature, animate and inanimate
• {{annotated link|Decision theory}}
• {{annotated link|Embodied cognition}}
• {{annotated link|Entropic force}}
• {{annotated link|Principle of minimum energy}}
• {{annotated link|Info-metrics}}
• {{annotated link|Optimal control}}
• {{annotated link|Adaptive system}}
• {{annotated link|Predictive coding}}
• {{annotated link|Self-organization}}
• {{annotated link|Surprisal}}
• {{annotated link|Synergetics (Haken)}}
• {{annotated link|Variational Bayesian methods}}

{{Reflist|3}}

• [http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=8918803 Behavioral and Brain Sciences (by Andy Clark)]

Category:Biological systems

Category:Systems theory

Category:Computational neuroscience

Category:Mathematical and theoretical biology