Chapter 13: Maxwell's Demon

Part V: Information & Physics

The Thought Experiment

In 1867, James Clerk Maxwell proposed a thought experiment to challenge the second law of thermodynamics. Imagine a gas-filled box divided in two. A tiny intelligent being — the "demon" — watches individual molecules and operates a frictionless trapdoor between the two halves.

The demon lets fast molecules pass to the right chamber and slow molecules to the left. Over time, the right chamber becomes hotter and the left cooler — without any work being done. Heat appears to flow spontaneously from cold to hot, violating the second law.

For nearly a century, this posed a serious challenge to thermodynamics. The resolution came from an unlikely direction: information theory.

Maxwell's Demon: Visual Schematic

The demon sorts fast (red) and slow (blue) molecules, maintaining a 1-bit memory register about each molecule's position.

The Szilard Engine (1929)

Leo Szilard simplified Maxwell's demon to a single molecule in a box. The engine operates in three steps:

Measurement: The demon measures which half of the box contains the molecule, gaining 1 bit of information.
Work extraction: A piston compresses the empty half isothermally. The molecule's pressure does work \( W = k_B T \ln 2 \) on the piston.
Reset: The partition is removed, ready for the next cycle.

If no entropy is paid, this constitutes a perpetual motion machine of the second kind. Szilard noted the measurement must cost entropy — but it took Landauer and Bennett to pinpoint exactly where.

Resolution: Information Erasure

The key insight came from Rolf Landauer (1961) and was completed by Charles Bennett (1982):

\( \Delta S_{\text{gas}} = -k_B \ln 2 \quad \text{(entropy decrease)} \)

\( \Delta S_{\text{erasure}} = +k_B \ln 2 \quad \text{(Landauer's principle)} \)

\( \Delta S_{\text{total}} = 0 \quad \checkmark \)

The demon's memory stores a bit: "molecule was on the LEFT" or "molecule was on the RIGHT." To operate cyclically, the demon must erase this bit before the next measurement. Landauer's principle says erasing one bit costs at least \(k_B T \ln 2\) of energy, dissipated as heat into the environment.

Bennett showed that the measurement itself can be done reversibly (zero entropy cost). The entire entropy payment happens at erasure. The second law survives because: information has a physical entropy equivalent.

Brillouin's Negentropy Principle

Léon Brillouin (1951) formalized the connection: acquiring one bit of information corresponds to a decrease in entropy of \(k_B \ln 2\). Conversely, erasing information increases entropy by the same amount.

\( I = -\frac{\Delta S}{k_B \ln 2} \quad \text{[bits]} \)

This "negentropy" (negative entropy) interpretation directly links Shannon's information measure to Boltzmann's thermodynamic entropy — a connection that Shannon himself noticed when von Neumann told him to call his measure "entropy."

Python: Szilard Engine Simulation

Simulate 1000 cycles of the single-molecule Szilard engine, tracking entropy accounting and confirming that the second law is always satisfied.

Python

script.py122 lines

import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

# Szilard Engine Simulation
# Single molecule in box: demon measures position, inserts partition, extracts work

np.random.seed(42)
N_cycles = 1000

# Each cycle: molecule starts in random position in [0,1]
# Demon measures which half, inserts partition, compresses isothermally -> extracts kT ln2
# Then demon must erase its 1-bit memory -> costs kT ln2

kB = 1.380649e-23  # J/K
T = 300.0          # K room temperature

W_extracted = kB * T * np.log(2)   # work extracted per cycle (J)
W_erasure   = kB * T * np.log(2)   # cost to erase demon's memory (Landauer)
W_net       = W_extracted - W_erasure  # net work: ZERO (second law upheld)

print(f"=== Szilard Engine Analysis ===")
print(f"Temperature: T = {T} K")
print(f"Work extracted per cycle:        W_out = kT ln2 = {W_extracted:.4e} J")
print(f"Energy cost to erase memory:     W_era = kT ln2 = {W_erasure:.4e} J")
print(f"Net work output:                 W_net = {W_net:.4e} J  (second law satisfied)")
print()

# Simulate entropy accounting over many cycles
cycles = np.arange(1, N_cycles + 1)

# Entropy decrease in gas per cycle: -kB ln2 (molecule confined to one half)
delta_S_gas = -kB * np.log(2)

# Entropy increase in environment during erasure: +kB ln2 (Landauer)
delta_S_env = kB * np.log(2)

# Cumulative entropy
cumulative_gas = delta_S_gas * cycles
cumulative_env = delta_S_env * cycles
cumulative_total = (delta_S_gas + delta_S_env) * cycles  # = 0 for reversible demon

print(f"Entropy change per cycle in gas:         {delta_S_gas:.4e} J/K")
print(f"Entropy change per cycle in environment: {delta_S_env:.4e} J/K")
print(f"Total entropy change per cycle:          {delta_S_gas + delta_S_env:.4e} J/K")
print()

# Simulate molecule positions across cycles
positions = np.random.uniform(0, 1, N_cycles)
in_left = positions < 0.5
fraction_left = np.cumsum(in_left) / cycles

print(f"Fraction of cycles molecule found in left half: {fraction_left[-1]:.3f} (expected 0.5)")
print()

# Show information-entropy equivalence
bits_of_info = 1.0  # demon gains 1 bit per measurement
entropy_equivalent = bits_of_info * kB * np.log(2)
print(f"Information gained per measurement: 1 bit = {entropy_equivalent:.4e} J/K")
print(f"This exactly equals the entropy decrease in the gas: {abs(delta_S_gas):.4e} J/K")
print(f"Brillouin's negentropy principle: I = -ΔS / (kB ln2) [bits]")

# Plot
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
fig.patch.set_facecolor('#0a0a1a')

# Panel 1: Entropy accounting
ax1 = axes[0]
ax1.plot(cycles, cumulative_gas / kB, color='#f87171', linewidth=2, label='Gas entropy (ΔS_gas)')
ax1.plot(cycles, cumulative_env / kB, color='#34d399', linewidth=2, label='Environment (erasure)')
ax1.plot(cycles, cumulative_total / kB, color='#818cf8', linewidth=2.5, linestyle='--', label='Total ΔS = 0')
ax1.set_xlabel('Cycle number', color='white', fontsize=11)
ax1.set_ylabel('Cumulative entropy change (units of kB)', color='white', fontsize=11)
ax1.set_title("Entropy Accounting\nSzilard Engine", color='white', fontsize=12, fontweight='bold')
ax1.legend(fontsize=8, facecolor='#1a1a2e', edgecolor='#818cf8', labelcolor='white')
ax1.set_facecolor('#0a0a1a')
ax1.tick_params(colors='white')
ax1.grid(True, alpha=0.2, color='#818cf8')
for sp in ax1.spines.values():
    sp.set_color('#818cf8')

# Panel 2: Molecule position histogram
ax2 = axes[1]
bins = np.linspace(0, 1, 25)
ax2.hist(positions[in_left], bins=bins, color='#818cf8', alpha=0.7, label='Left half (demon inserts right partition)')
ax2.hist(positions[~in_left], bins=bins, color='#a5b4fc', alpha=0.7, label='Right half (demon inserts left partition)')
ax2.axvline(0.5, color='#fbbf24', linewidth=2, linestyle='--', label='Partition position')
ax2.set_xlabel('Molecule position', color='white', fontsize=11)
ax2.set_ylabel('Count', color='white', fontsize=11)
ax2.set_title("Molecule Position Distribution\n(1000 cycles)", color='white', fontsize=12, fontweight='bold')
ax2.legend(fontsize=8, facecolor='#1a1a2e', edgecolor='#818cf8', labelcolor='white')
ax2.set_facecolor('#0a0a1a')
ax2.tick_params(colors='white')
ax2.grid(True, alpha=0.2, color='#818cf8')
for sp in ax2.spines.values():
    sp.set_color('#818cf8')

# Panel 3: Work budget
ax3 = axes[2]
categories = ['Work\nextracted', 'Erasure\ncost', 'Net\noutput']
values = [W_extracted * 1e21, W_erasure * 1e21, W_net * 1e21]
colors_bar = ['#34d399', '#f87171', '#818cf8']
bars = ax3.bar(categories, values, color=colors_bar, alpha=0.85, edgecolor='white', linewidth=0.8)
ax3.axhline(0, color='white', linewidth=1)
ax3.set_ylabel('Energy (×10⁻²¹ J)', color='white', fontsize=11)
ax3.set_title("Work Budget per Cycle\n(T = 300 K)", color='white', fontsize=12, fontweight='bold')
ax3.set_facecolor('#0a0a1a')
ax3.tick_params(colors='white')
ax3.grid(True, alpha=0.2, color='#818cf8', axis='y')
for sp in ax3.spines.values():
    sp.set_color('#818cf8')
for bar, val in zip(bars, values):
    ax3.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.05,
             f'{val:.2f}', ha='center', va='bottom', color='white', fontsize=10)

plt.tight_layout()
plt.savefig('output.png', dpi=150, bbox_inches='tight', facecolor='#0a0a1a')
plt.show()
print("\nConclusion: The demon can decrease gas entropy but must erase its memory,")
print("costing exactly kT ln2 per bit — the second law is always satisfied.")

Click Run to execute the Python code

Code will be executed with Python 3 on the server

← Ch 12: Gödel, Turing & Information Ch 14: Landauer's Principle →