Part III › Chapter 8

Gaussian Channel Capacity

The Shannon-Hartley theorem gives the fundamental limit of every analog communication channel. Every hertz of bandwidth and every watt of power has a precise informational value.

8.1 The AWGN Channel

The Additive White Gaussian Noise (AWGN) channel is the canonical continuous-channel model:

\[ Y = X + Z, \qquad Z \sim \mathcal{N}(0, N) \]

with a power constraint\(E[X^2] \le P\). The noise \(Z\) is independent of the input \(X\). The goal: find the input distribution \(p^*(x)\) that maximizes mutual information \(I(X;Y)\).

Key step: \(I(X;Y) = h(Y) - h(Y|X) = h(Y) - h(Z)\). Since \(h(Z) = \frac{1}{2}\log(2\pi e N)\)is fixed, we need to maximize \(h(Y)\). The variance of \(Y\) is\(\operatorname{Var}(Y) = P + N\). By the max-entropy theorem, \(h(Y)\)is maximized when \(Y\) is Gaussian, which occurs when \(X\) is Gaussian.

8.2 Shannon-Hartley Theorem

For a channel of bandwidth \(B\) Hz with signal power \(S\)and noise power \(N\):

\[ C = B\log_2\!\left(1 + \frac{S}{N}\right) \quad\text{[bits/second]} \]

This is the capacity of the AWGN channel — the maximum rate at which information can be transmitted with arbitrarily small error probability. The optimal input distribution is\(X \sim \mathcal{N}(0, P)\).

High SNR

\[ C \approx B\log_2\!\frac{S}{N} \]

Each 3 dB of SNR adds ~1 bit/s/Hz

Low SNR (wideband)

\[ C \approx \frac{S}{N_0 \ln 2} \]

Capacity proportional to power, not bandwidth

Shannon limit

\[ \frac{E_b}{N_0} \ge \ln 2 \approx -1.59 \;\text{dB} \]

Minimum energy per bit for reliable comm.

8.3 The Shannon Limit: −1.6 dB

The Shannon limit is the minimum energy per bit\(E_b/N_0\) required for reliable communication, regardless of bandwidth. Define\(R\) as the code rate in bits/s and \(E_b = P/R\) as energy per bit:

\[ C = B\log_2\!\left(1 + \frac{E_b R}{N_0 B}\right) \]

For \(R \le C\) and taking \(B\to\infty\) (infinite bandwidth):

\[ \frac{E_b}{N_0} \ge \ln 2 \approx 0.693 \approx -1.59\;\text{dB} \]

This is an absolute floor — no coding scheme, however complex, can communicate reliably below this threshold. Modern codes (turbo, LDPC, polar) operate within 0.1 dB of this limit.

8.4 Bandwidth–Power Tradeoff

Bandwidth and power are interchangeable commodities for achieving a target capacity. Rewrite with noise spectral density \(N = N_0 B\):

\[ C = B\log_2\!\left(1 + \frac{S}{N_0 B}\right) \]

‣Fixed S, increase B: capacity grows, but with diminishing returns (converges to \(S/N_0\ln 2\))
‣Fixed B, double S: capacity increases by less than 1 bit/s/Hz at high SNR (logarithmic)
‣Wideband regime (low SNR): capacity \(\propto S\) — doubling power doubles capacity
‣Bandwidth-limited: more power helps; Power-limited: more bandwidth helps

Python: Shannon-Hartley Analysis

Four plots: capacity vs SNR with real system markers, the bandwidth-power tradeoff curve, practical systems mapped against the Shannon bound, and mutual information as a function of input power for varying noise levels.

Python

script.py150 lines

import numpy as np
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches

plt.style.use('dark_background')

fig, axes = plt.subplots(2, 2, figsize=(13, 10))
fig.patch.set_facecolor('#0a0a0f')
for ax in axes.flat:
    ax.set_facecolor('#0d0d1a')

# ── 1. Shannon-Hartley: Capacity vs SNR ───────────────────────────────────
snr_db = np.linspace(-10, 40, 500)
snr_lin = 10 ** (snr_db / 10)
B = 1.0  # normalize bandwidth to 1 Hz

C = B * np.log2(1 + snr_lin)  # bits/s/Hz

ax = axes[0, 0]
ax.plot(snr_db, C, color='#818cf8', lw=2.5, label='C = log₂(1+SNR)')
ax.axvline(-1.59, color='#f87171', lw=1.5, ls='--', label='Shannon limit  −1.6 dB')
ax.axhline(0, color='#6b7280', lw=0.8, ls='--', alpha=0.5)

# Practical points
systems = [
    ('WiFi 802.11n', 25, 7.5, '#34d399'),
    ('4G LTE',       15, 5.0, '#fbbf24'),
    ('5G NR',        30, 9.9, '#a78bfa'),
    ('DSL',           8, 3.2, '#f59e0b'),
]
for name, snr_d, cap_approx, col in systems:
    c_val = np.log2(1 + 10**(snr_d/10))
    ax.scatter([snr_d], [c_val], color=col, s=70, zorder=5)
    ax.text(snr_d + 0.8, c_val + 0.1, name, color=col, fontsize=8)

ax.set_xlabel('SNR  [dB]', color='#9ca3af')
ax.set_ylabel('Spectral Efficiency  [bit/s/Hz]', color='#9ca3af')
ax.set_title('Shannon-Hartley: C vs SNR', color='#a5b4fc', fontweight='bold')
ax.legend(fontsize=9, facecolor='#0d0d1a', edgecolor='#374151', labelcolor='white')
ax.tick_params(colors='#6b7280')
for spine in ax.spines.values(): spine.set_edgecolor('#1e293b')

# ── 2. Bandwidth–Power tradeoff ────────────────────────────────────────────
ax = axes[0, 1]
S = 1.0  # fixed total power
N0 = 0.1  # noise spectral density

B_vals = np.linspace(0.1, 20, 500)
# C = B log2(1 + S/(N0*B))
C_bw = B_vals * np.log2(1 + S / (N0 * B_vals))

# Shannon limit: C → S/(N0 ln2) as B → ∞
C_limit = S / (N0 * np.log(2))

ax.plot(B_vals, C_bw, color='#818cf8', lw=2.5, label='C = B log₂(1+S/N₀B)')
ax.axhline(C_limit, color='#f87171', lw=1.5, ls='--',
           label=f'Wideband limit  C→S/N₀ln2 = {C_limit:.2f} bit/s')

# Annotations at specific bandwidths
for B_mark in [1, 5, 10]:
    c_mark = B_mark * np.log2(1 + S/(N0*B_mark))
    ax.scatter([B_mark], [c_mark], color='#34d399', s=50, zorder=5)
    ax.text(B_mark + 0.3, c_mark - 0.4, f'B={B_mark}', color='#34d399', fontsize=8)

ax.set_xlabel('Bandwidth B  [Hz]', color='#9ca3af')
ax.set_ylabel('Capacity C  [bit/s]', color='#9ca3af')
ax.set_title('Bandwidth–Power Tradeoff (fixed power)', color='#a5b4fc', fontweight='bold')
ax.legend(fontsize=9, facecolor='#0d0d1a', edgecolor='#374151', labelcolor='white')
ax.tick_params(colors='#6b7280')
for spine in ax.spines.values(): spine.set_edgecolor('#1e293b')

# ── 3. Practical systems on C–SNR plane ───────────────────────────────────
ax = axes[1, 0]
snr_fine = np.linspace(-5, 45, 600)
C_curve = np.log2(1 + 10**(snr_fine/10))
ax.plot(snr_fine, C_curve, color='#818cf8', lw=2.5, alpha=0.7, label='Shannon bound')
ax.fill_between(snr_fine, 0, C_curve, alpha=0.07, color='#818cf8')

practical_systems = [
    # (name, SNR_dB, spectral_eff, color, marker)
    ('BPSK (uncoded)',   5,   1.0,  '#94a3b8', 'o'),
    ('QPSK',             7,   2.0,  '#94a3b8', 's'),
    ('16-QAM',          13,   4.0,  '#94a3b8', '^'),
    ('64-QAM',          20,   6.0,  '#94a3b8', 'D'),
    ('256-QAM',         26,   8.0,  '#94a3b8', 'v'),
    ('WiFi 802.11ac',   28,   8.7,  '#34d399', 'o'),
    ('WiFi 802.11ax',   30,   9.6,  '#34d399', 's'),
    ('4G LTE Cat 15',   22,   7.5,  '#fbbf24', 'o'),
    ('5G mmWave',       35,  10.5,  '#a78bfa', 'o'),
    ('5G sub-6GHz',     28,   8.9,  '#a78bfa', 's'),
    ('Fiber (coherent)', 30, 10.0,  '#f87171', 'o'),
]
legend_patches = []
colors_seen = {}
for name, snr_d, se, col, mk in practical_systems:
    ax.scatter([snr_d], [se], color=col, marker=mk, s=80, zorder=5, edgecolors='white', lw=0.5)
    ax.text(snr_d + 0.4, se, name, color=col, fontsize=7, va='center')
    if col not in colors_seen:
        colors_seen[col] = name.split()[0] if name.split()[0] != 'WiFi' else 'WiFi'
        legend_patches.append(mpatches.Patch(color=col, label=colors_seen[col]))

ax.axvline(-1.59, color='#f87171', lw=1.2, ls=':', alpha=0.6)
ax.set_xlabel('SNR  [dB]', color='#9ca3af')
ax.set_ylabel('Spectral Efficiency  [bit/s/Hz]', color='#9ca3af')
ax.set_title('Practical Systems vs Shannon Bound', color='#a5b4fc', fontweight='bold')
ax.legend(handles=legend_patches + [mpatches.Patch(color='#818cf8', label='Shannon bound')],
          fontsize=8, facecolor='#0d0d1a', edgecolor='#374151', labelcolor='white', loc='upper left')
ax.tick_params(colors='#6b7280')
ax.set_xlim(-5, 45); ax.set_ylim(-0.2, 13)
for spine in ax.spines.values(): spine.set_edgecolor('#1e293b')

# ── 4. Derivation: mutual information maximization ────────────────────────
ax = axes[1, 1]
sigma2_list = [0.25, 0.5, 1.0, 2.0, 4.0]
colors_mi = ['#818cf8','#34d399','#fbbf24','#f87171','#a78bfa']
snr_x = np.linspace(0, 50, 300)

for s2, col in zip(sigma2_list, colors_mi):
    # I(X;Y) = h(Y) - h(N) = 0.5 log2(1 + P/sigma2)
    # P = snr_x for normalized
    C_vals = 0.5 * np.log2(1 + snr_x / s2)
    ax.plot(snr_x, C_vals, color=col, lw=2,
            label=f'σ²_N = {s2}')

ax.set_xlabel('Input Power P', color='#9ca3af')
ax.set_ylabel('I(X;Y)  [bits]', color='#9ca3af')
ax.set_title('Capacity = ½ log₂(1+P/σ²): varying noise', color='#a5b4fc', fontweight='bold')
ax.legend(fontsize=8, facecolor='#0d0d1a', edgecolor='#374151', labelcolor='white')
ax.tick_params(colors='#6b7280')
ax.text(28, 0.4,
        'I(X;Y) = h(Y) − h(N)\nMax over p(x): Y Gaussian → h(Y) max',
        color='#9ca3af', fontsize=8,
        bbox=dict(boxstyle='round,pad=0.4', facecolor='#0d0d1a', edgecolor='#374151'))
for spine in ax.spines.values(): spine.set_edgecolor('#1e293b')

fig.suptitle('Chapter 8: Gaussian Channel Capacity & Shannon-Hartley Theorem',
             color='#a5b4fc', fontsize=14, fontweight='bold', y=1.01)
plt.tight_layout()
plt.savefig('output.png', dpi=130, bbox_inches='tight',
            facecolor='#0a0a0f', edgecolor='none')
print("Saved output.png")

# Print a few key values
for snr_d in [-1.59, 0, 3, 10, 20, 30]:
    c = np.log2(1 + 10**(snr_d/10))
    print(f"  SNR = {snr_d:+.2f} dB  →  C = {c:.3f} bit/s/Hz")
print(f"\nWideband limit S/N₀ln2 = {1/(0.1*np.log(2)):.3f} bit/s  (S=1, N₀=0.1)")

Click Run to execute the Python code

Code will be executed with Python 3 on the server

← Differential Entropy Next: MIMO & Water-Filling →