Chapter 16: Von Neumann Entropy

Part VI: Quantum Information

Definition

$ S(\rho) = -\text{Tr}(\rho \ln \rho) = -\sum_i \lambda_i \ln \lambda_i $

where $\lambda_i$ are the eigenvalues of the density matrix $\rho$

John von Neumann introduced this measure in 1927 as the quantum analog of the Boltzmann entropy. For a quantum system in state $\rho$ (a positive semidefinite Hermitian operator with $\text{Tr}(\rho) = 1$):

For pure states $\rho = |\psi\rangle\langle\psi|$: $S(\rho) = 0$ (no uncertainty)
For the maximally mixed state $\rho = I/d$: $S(\rho) = \ln d$ (maximum entropy)
In bits: divide by $\ln 2$

Relation to Shannon Entropy

The Von Neumann entropy reduces to Shannon entropy when the density matrix is diagonal:

$ \rho = \sum_i p_i |i\rangle\langle i| \implies S(\rho) = -\sum_i p_i \ln p_i = H(\{p_i\}) $

The key difference: quantum systems can be in coherent superpositions, and entropy depends on the basis. The Von Neumann entropy is basis-independent (it only depends on eigenvalues), measuring genuine quantum uncertainty.

Measuring a quantum system in a specific basis increases entropy (collapses superpositions), while coherent evolution (unitary operations) preserves it.

Key Properties

Subadditivity

$ S(AB) \leq S(A) + S(B) $

Joint entropy never exceeds sum of individual entropies. Equality iff $\rho_{AB} = \rho_A \otimes \rho_B$ (product state).

Strong Subadditivity (SSA)

$ S(ABC) + S(B) \leq S(AB) + S(BC) $

Proven by Lieb and Ruskai (1973). One of the most important inequalities in quantum information theory.

Araki-Lieb Triangle Inequality

$ S(AB) \geq |S(A) - S(B)| $

Unlike classical entropy, a quantum subsystem can have more entropy than the whole system (when entangled).

Unitary Invariance

$ S(U\rho U^\dagger) = S(\rho) $ for any unitary $U$

Reversible quantum operations preserve entropy; only irreversible operations (measurements, decoherence) change it.

Entanglement Entropy & Purification

For a bipartite system $AB$ in a pure state, the entanglement entropy is the Von Neumann entropy of either reduced density matrix:

$ E(|\psi\rangle_{AB}) = S(\rho_A) = S(\rho_B) \quad \text{where } \rho_A = \text{Tr}_B(|\psi\rangle\langle\psi|) $

Purification: Any mixed state $\rho_A$can be "purified" — embedded as a pure state in a larger system $AB$such that $\text{Tr}_B(|\psi\rangle\langle\psi|) = \rho_A$. This is central to the Church of the Larger Hilbert Space (Stinespring dilation theorem).

Entanglement entropy quantifies quantum correlations between $A$ and $B$. For a Bell state $|\Phi^+\rangle = (|00\rangle + |11\rangle)/\sqrt{2}$,$S(\rho_A) = 1$ bit — maximal entanglement.

Python: Von Neumann Entropy Computations

Compute entropies for pure states, maximally mixed states, Werner states, and bipartite systems. Plot entanglement entropy vs mixing parameter.

Python

script.py142 lines

import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt

# Von Neumann entropy: S(rho) = -Tr(rho ln rho) = -sum_i lambda_i ln lambda_i
# where lambda_i are eigenvalues of the density matrix

def von_neumann_entropy(rho):
    """Compute Von Neumann entropy (nats) of density matrix rho."""
    eigenvalues = np.linalg.eigvalsh(rho)
    eigenvalues = eigenvalues[eigenvalues > 1e-15]  # remove near-zero
    return -np.sum(eigenvalues * np.log(eigenvalues))

def von_neumann_bits(rho):
    return von_neumann_entropy(rho) / np.log(2)

# ── Pure states ──────────────────────────────────────────────────────────────
# |0><0|
rho_pure0 = np.array([[1, 0], [0, 0]], dtype=complex)
# |+><+| = (|0>+|1>)/sqrt(2)
rho_plus = np.array([[0.5, 0.5], [0.5, 0.5]], dtype=complex)
# Maximally mixed (I/2)
rho_mixed = np.eye(2) / 2

print("=== Von Neumann Entropy ===")
print(f"Pure state |0>:          S = {von_neumann_bits(rho_pure0):.4f} bits")
print(f"Pure state |+>:          S = {von_neumann_bits(rho_plus):.4f} bits")
print(f"Maximally mixed (I/2):   S = {von_neumann_bits(rho_mixed):.4f} bits  (max = 1 bit)")
print()

# ── Werner states rho = p|Phi+><Phi+| + (1-p)I/4 ────────────────────────────
# |Phi+> = (|00>+|11>)/sqrt(2) (maximally entangled Bell state)
# Werner states: p=1 -> pure entangled, p=0 -> maximally mixed
def werner_state(p):
    """2-qubit Werner state."""
    phi_plus = np.array([1,0,0,1]) / np.sqrt(2)
    rho_bell = np.outer(phi_plus, phi_plus.conj())
    rho_id   = np.eye(4) / 4
    return p * rho_bell + (1 - p) * rho_id

def entanglement_entropy_werner(p):
    """Entanglement entropy of Werner state (entropy of reduced density matrix)."""
    rho_AB = werner_state(p)
    # Partial trace over B (trace out second qubit)
    rho_A = np.array([
        [rho_AB[0,0] + rho_AB[1,1], rho_AB[0,2] + rho_AB[1,3]],
        [rho_AB[2,0] + rho_AB[3,1], rho_AB[2,2] + rho_AB[3,3]]
    ], dtype=complex)
    return von_neumann_bits(rho_A)

p_vals = np.linspace(0, 1, 200)
S_A_vals = [entanglement_entropy_werner(p) for p in p_vals]
S_AB_vals = [von_neumann_bits(werner_state(p)) for p in p_vals]

print("Werner state S(rho_A) (entanglement entropy) at key points:")
for p_test in [0.0, 0.25, 0.5, 0.75, 1.0]:
    idx = np.argmin(np.abs(p_vals - p_test))
    print(f"  p = {p_test}: S(A) = {S_A_vals[idx]:.4f} bits, S(AB) = {S_AB_vals[idx]:.4f} bits")
print()

# ── Bipartite entanglement entropy for |psi> = sqrt(1-x)|00> + sqrt(x)|11> ─
x_vals = np.linspace(0, 1, 300)
S_ent = []
for x in x_vals:
    if x < 1e-12 or x > 1 - 1e-12:
        S_ent.append(0.0)
    else:
        lambdas = np.array([1 - x, x])
        S_ent.append(-np.sum(lambdas * np.log2(lambdas)))

print(f"Entanglement entropy of |psi> = sqrt(1-x)|00>+sqrt(x)|11>:")
print(f"  x=0.0: S=0.000 (product state)")
print(f"  x=0.5: S={-2*0.5*np.log2(0.5):.3f} bit (maximally entangled)")
print(f"  x=1.0: S=0.000 (product state)")

# ── Subadditivity check ──────────────────────────────────────────────────────
print()
print("Subadditivity check: S(AB) <= S(A) + S(B)")
for p_test in [0.2, 0.5, 0.8]:
    idx = np.argmin(np.abs(p_vals - p_test))
    S_AB = S_AB_vals[idx]
    S_A  = S_A_vals[idx]
    S_B  = S_A_vals[idx]  # symmetric Werner state
    print(f"  p={p_test}: S(AB)={S_AB:.3f}, S(A)+S(B)={S_A+S_B:.3f} -> {'OK' if S_AB <= S_A+S_B+1e-9 else 'VIOLATED'}")

# ── Plotting ──────────────────────────────────────────────────────────────────
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
fig.patch.set_facecolor('#0a0a1a')

# Plot 1: Entanglement entropy vs x for bipartite pure state
ax1 = axes[0]
ax1.plot(x_vals, S_ent, color='#818cf8', linewidth=2.5)
ax1.axvline(0.5, color='#fbbf24', linestyle='--', linewidth=1.5, label='Max entanglement (x=0.5)')
ax1.set_xlabel('Mixing parameter x', color='white', fontsize=11)
ax1.set_ylabel('Entanglement entropy (bits)', color='white', fontsize=11)
ax1.set_title("Entanglement Entropy\n$|\\psi\\rangle = \\sqrt{1-x}|00\\rangle + \\sqrt{x}|11\\rangle$",
              color='white', fontsize=12, fontweight='bold')
ax1.legend(fontsize=9, facecolor='#1a1a2e', edgecolor='#818cf8', labelcolor='white')
ax1.set_facecolor('#0a0a1a')
ax1.tick_params(colors='white')
ax1.grid(True, alpha=0.2, color='#818cf8')
for sp in ax1.spines.values(): sp.set_color('#818cf8')

# Plot 2: Werner state entropies vs p
ax2 = axes[1]
ax2.plot(p_vals, S_A_vals,  color='#818cf8', linewidth=2.5, label='S(A) = entanglement entropy')
ax2.plot(p_vals, S_AB_vals, color='#a5b4fc', linewidth=2, linestyle='--', label='S(AB) = joint entropy')
ax2.axvline(1/3, color='#f87171', linestyle=':', linewidth=1.5, alpha=0.8, label='p=1/3 (separability threshold)')
ax2.set_xlabel('Werner parameter p', color='white', fontsize=11)
ax2.set_ylabel('Von Neumann entropy (bits)', color='white', fontsize=11)
ax2.set_title("Werner State Entropies\n$\\rho_W = p|\\Phi^+\\rangle\\langle\\Phi^+| + (1-p)I/4$",
              color='white', fontsize=12, fontweight='bold')
ax2.legend(fontsize=9, facecolor='#1a1a2e', edgecolor='#818cf8', labelcolor='white')
ax2.set_facecolor('#0a0a1a')
ax2.tick_params(colors='white')
ax2.grid(True, alpha=0.2, color='#818cf8')
for sp in ax2.spines.values(): sp.set_color('#818cf8')

# Plot 3: Entropy for a qubit state rho = (1-p)|0><0| + p|1><1|
p_qubit = np.linspace(0, 1, 300)
S_qubit = np.where((p_qubit < 1e-12) | (p_qubit > 1 - 1e-12), 0,
                   -p_qubit * np.log2(p_qubit) - (1 - p_qubit) * np.log2(1 - p_qubit))
ax3 = axes[2]
ax3.plot(p_qubit, S_qubit, color='#818cf8', linewidth=2.5, label='Von Neumann entropy')
ax3.fill_between(p_qubit, S_qubit, alpha=0.15, color='#818cf8')
ax3.axvline(0.5, color='#fbbf24', linestyle='--', linewidth=1.5, label='Max entropy (maximally mixed)')
ax3.set_xlabel('Mixture parameter p', color='white', fontsize=11)
ax3.set_ylabel('S(ρ) (bits)', color='white', fontsize=11)
ax3.set_title("Single Qubit Entropy\n$\\rho = (1-p)|0\\rangle\\langle 0| + p|1\\rangle\\langle 1|$",
              color='white', fontsize=12, fontweight='bold')
ax3.legend(fontsize=9, facecolor='#1a1a2e', edgecolor='#818cf8', labelcolor='white')
ax3.set_facecolor('#0a0a1a')
ax3.tick_params(colors='white')
ax3.grid(True, alpha=0.2, color='#818cf8')
for sp in ax3.spines.values(): sp.set_color('#818cf8')

plt.tight_layout()
plt.savefig('output.png', dpi=150, bbox_inches='tight', facecolor='#0a0a1a')
plt.show()
print("\nS(rho)=0 iff rho is a pure state; S(rho)=log d iff rho = I/d (maximally mixed).")

Click Run to execute the Python code

Code will be executed with Python 3 on the server

← Ch 15: Black Hole Information Ch 17: Quantum Channels →