Generative Models of the World

1. Helmholtz’s Unconscious Inference

Helmholtz observed that visual perception routinely adds information not present in the retinal image — depth from binocular disparity, object permanence across saccades, illumination correction for reflectance. He framed this as unconscious inversion of a generative model: the brain asks “what causes would have produced these sensations?” and returns the most probable answer. Roughly 150 years later, that idea was formalised as Bayesian perception (Knill & Pouget 2004).

2. Hierarchical Latent Variables

A world model is a hierarchy of latent variables where each level generates the level below:

\[ P(\text{sensation}) \;=\; \int P(\text{sensation}\mid z_1)\,P(z_1\mid z_2)\,P(z_2\mid z_3)\cdots dz_1\,dz_2\,\cdots \]

In vision, low-level latents are edges and textures; mid-level are objects; high-level are scenes and abstract concepts. Cortex is organised hierarchically along this gradient. Rao & Ballard 1999 showed that a specific generative- model hierarchy reproduces V1 receptive-field properties when trained on natural images. M3 covers predictive coding in detail.

3. Inference in the Brain

Exact Bayesian inference is generally intractable for high-dimensional world-models. The brain must approximate. Two major routes considered in later modules: (a) variational inference (M2, M4) — approximate the posterior with a tractable family; (b) sampling (not covered in depth) — explicit Monte-Carlo over latents. Predictive coding (M3) implements a particular form of variational inference that maps onto cortical microcircuits.

Simulation: 2-D Latent, High-D Observation

Key References