Module 4

The Free-Energy Principle

Friston’s free-energy principle (FEP) proposes that any self-organising system that persists must minimise its variational free energy — an upper bound on surprise. FEP unifies perception, action, learning, and homeostasis under a single imperative. This module covers the mathematical core and the Markov- blanket formulation.

1. Surprise and Its Bound

Surprise (surprisal) is −log P(x): the negative log probability of an observation under the generative model. Direct minimisation requires integrating over latents. Variational free energy provides a tractable upper bound:

\[ F \;=\; \mathbb{E}_{Q}[\log Q(z) - \log P(x, z)] \;=\; D_{KL}[Q\|P(z|x)] - \log P(x) \]

Because KL divergence is non-negative, F ≥ −log P(x). Minimising F simultaneously (a) pushes Q toward the true posterior and (b) reduces surprise. This is the same ELBO from M2, with the sign flipped and the perspective changed.

2. Markov Blankets & Self-Organisation

Friston 2013 and 2019 extended the FEP using Markov-blanket formalism: a system is defined by the partition of its state into internal, external, and blanket (sensory + active) states, with conditional independence across the blanket. Any system that exists as a distinct entity implicitly has a Markov blanket, and the dynamics of internal states that maintain the partition correspond to free-energy minimisation. This is the controversial but far-reaching claim that life itself is Bayesian inference.

Simulation: Variational Free Energy

Python

script.py45 lines

import numpy as np, matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

# Variational free energy F = D_KL[Q || P(z|x)] - log P(x)
# Minimising F simultaneously minimises KL and maximises log-evidence
x = 2.0
prior_mean, prior_std = 0.0, 1.0
sensor_std = 0.5
mu_range = np.linspace(-2, 4, 300)
# Approximate posterior Q ~ N(mu, 0.3)
Q_std = 0.3

def free_energy(mu):
    # prior negative log
    nlp = 0.5*(mu - prior_mean)**2 / prior_std**2 + 0.5*np.log(2*np.pi*prior_std**2)
    # likelihood negative log
    nll = 0.5*(x - mu)**2 / sensor_std**2 + 0.5*np.log(2*np.pi*sensor_std**2)
    # entropy of Q
    H = 0.5*np.log(2*np.pi*np.e*Q_std**2)
    return nlp + nll - H

F = np.array([free_energy(m) for m in mu_range])
mu_star = mu_range[np.argmin(F)]

# True posterior mean (conjugate Gaussian)
post_var = 1 / (1/prior_std**2 + 1/sensor_std**2)
post_mean = post_var * (prior_mean/prior_std**2 + x/sensor_std**2)

fig, ax = plt.subplots(figsize=(11, 5), facecolor='#0a0a1a')
ax.set_facecolor('#111827'); ax.tick_params(colors='#cbd5e1')
for s in ax.spines.values(): s.set_color('#334155')
ax.plot(mu_range, F, color='#a78bfa', lw=2.5)
ax.axvline(mu_star, color='#fbbf24', ls='--', label=f'argmin F = {mu_star:.2f}')
ax.axvline(post_mean, color='#34d399', ls='--', label=f'true post mean = {post_mean:.2f}')
ax.set_xlabel('Variational posterior mean mu', color='#cbd5e1')
ax.set_ylabel('Free energy F(mu)', color='#cbd5e1')
ax.set_title('Free-energy minimisation = Bayesian inference',
             color='#c4b5fd', fontweight='bold')
ax.legend(facecolor='#1e293b', edgecolor='#334155', labelcolor='#cbd5e1')
ax.grid(True, color='#334155', alpha=0.3)
plt.tight_layout()
plt.savefig('output.png', dpi=120, bbox_inches='tight', facecolor='#0a0a1a')
print('Free energy = KL(Q||P(z|x)) - log P(x)')
print('Minimum aligns with true posterior mean')

Click Run to execute the Python code

Code will be executed with Python 3 on the server

3. Objections & Criticisms

FEP is controversial: critics (Colombo 2021, van Es 2020) argue that the mathematical generality comes at the cost of empirical content — any self-maintaining system trivially minimises free energy by construction, so the principle may be more descriptive than predictive. Proponents counter that the value is in the common currency it provides across perception, action, and learning, and in the falsifiable computational models (predictive coding, active inference) the principle generates.

Key References

• Friston, K. J. (2010). “The free-energy principle: a unified brain theory?” Nat. Rev. Neurosci., 11, 127–138.

• Friston, K. et al. (2013). “Life as we know it.” J. R. Soc. Interface, 10, 20130475.

• Parr, T., Pezzulo, G. & Friston, K. J. (2022). Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press.

• Colombo, M. & Palacios, P. (2021). “Non-equilibrium thermodynamics and the free-energy principle in biology.” Biol. Philos., 36, 41.

Share:X Reddit LinkedIn

← Module 3 Module 5: Active Inference →