Chapter 5: Sampling & the Nyquist Theorem
Sampling bridges the continuous and discrete worlds. The Shannon–Nyquist theorem tells us exactly how fast we must sample an analog signal to preserve all of its information, while sinc interpolation shows us how to perfectly reconstruct the original waveform from its samples. This chapter covers the full pipeline from ideal mathematical sampling to practical ADC/DAC systems.
5.1 — Ideal Sampling
Ideal sampling models the act of measuring a continuous-time signal \( x(t) \) at uniformly spaced instants \( t = nT_s \), where \( T_s = 1/f_s \) is the sampling period. Mathematically we represent this with the Dirac comb (also called the shah function or impulse train).
Definition: Dirac Comb (Impulse Train)
The Dirac comb with period \( T_s \) is
\[ \text{III}_{T_s}(t) = \sum_{n=-\infty}^{\infty} \delta(t - nT_s) \]
Multiplying by a signal \( x(t) \) produces the ideally sampled signal:
\[ x_s(t) = x(t) \cdot \text{III}_{T_s}(t) = \sum_{n=-\infty}^{\infty} x(nT_s)\,\delta(t - nT_s) \]
Spectral Replication
Multiplication in time corresponds to convolution in frequency. The Fourier transform of the Dirac comb is itself a Dirac comb:
\[ \mathcal{F}\{\text{III}_{T_s}(t)\} = f_s \cdot \text{III}_{f_s}(f) = f_s \sum_{k=-\infty}^{\infty} \delta(f - kf_s) \]
Therefore the spectrum of the sampled signal is a sum of shifted copies of the original spectrum:
\[ X_s(f) = f_s \sum_{k=-\infty}^{\infty} X(f - kf_s) \]
Remark: Why Copies?
Each term \( X(f - kf_s) \) is a replica of the baseband spectrum centered at the integer multiple \( kf_s \). If \( f_s \) is large enough these replicas do not overlap and the original \( X(f) \) can be recovered by an ideal low-pass filter. If they do overlap, aliasing occurs and information is irreversibly lost.
5.2 — Shannon–Nyquist Sampling Theorem
Theorem (Shannon–Nyquist Sampling Theorem)
Let \( x(t) \) be a band-limited signal whose Fourier transform satisfies\( X(f) = 0 \) for all \( |f| > f_B \). Then \( x(t) \) is completely determined by its samples \(\{x(nT_s)\}_{n \in \mathbb{Z}}\)provided the sampling rate satisfies
\[ f_s > 2f_B \]
The critical rate \( f_{\text{Nyquist}} = 2f_B \) is called the Nyquist rate, and \( f_B \) is the Nyquist frequency (i.e., the maximum frequency that can be faithfully represented at sampling rate \( f_s \) is \( f_s/2 \)).
Proof Sketch
From Section 5.1 the spectrum of the sampled signal is\( X_s(f) = f_s \sum_{k} X(f - kf_s) \). Because \( X(f) \) is supported on \( [-f_B, f_B] \), each replica \( X(f - kf_s) \) is supported on \( [kf_s - f_B,\; kf_s + f_B] \).
The baseband copy (\( k=0 \)) and its nearest neighbors (\( k = \pm 1 \)) will not overlap provided
\[ f_B < f_s - f_B \quad\Longleftrightarrow\quad f_s > 2f_B \]
When this condition holds, we can recover \( X(f) \) exactly by applying an ideal low-pass filter \( H(f) = T_s \cdot \text{rect}(f / f_s) \), which keeps only the\( k=0 \) replica and scales it back to the original amplitude. Inverse Fourier transforming this recovery procedure yields the sinc interpolation formula (Section 5.4).
Definition: Nyquist Rate vs. Nyquist Frequency
- Nyquist rate of a signal with bandwidth \( f_B \): the minimum sampling rate \( 2f_B \) required to avoid aliasing.
- Nyquist frequency of a system sampling at \( f_s \): the highest signal frequency it can represent, equal to \( f_s / 2 \).
Example: CD Audio
The compact disc standard samples at \( f_s = 44{,}100 \) Hz. Human hearing extends to roughly \( 20{,}000 \) Hz, so the Nyquist rate is \( 2 \times 20{,}000 = 40{,}000 \) Hz. The extra \( 4{,}100 \) Hz of headroom allows a practical anti-aliasing filter to roll off smoothly rather than requiring an infinitely steep brick-wall response.
5.3 — Aliasing
When the sampling rate violates the Nyquist condition (\( f_s < 2f_B \)), the spectral replicas overlap. High-frequency components “fold back” into lower frequencies, creating false signals indistinguishable from genuine low-frequency content. This phenomenon is called aliasing.
Definition: Alias Frequency
A tone at frequency \( f_0 \) sampled at rate \( f_s \) appears at the alias frequency
\[ f_{\text{alias}} = \bigl| f_0 - f_s \cdot \text{round}(f_0 / f_s) \bigr| \]
Equivalently, \( f_{\text{alias}} \) is the unique frequency in\( [0, f_s/2] \) such that\( \sin(2\pi f_0 t)\big|_{t=nT_s} = \pm\sin(2\pi f_{\text{alias}} \cdot nT_s) \).
Example: 5 Hz Tone Sampled at 8 Hz
Let \( f_0 = 5 \) Hz and \( f_s = 8 \) Hz. The Nyquist frequency is 4 Hz, so 5 Hz exceeds it. Computing:
\[ f_{\text{alias}} = |5 - 8 \cdot 1| = |5 - 8| = 3 \text{ Hz} \]
A 3 Hz sinusoid fits the same sample values as the original 5 Hz sinusoid. Without prior knowledge there is no way to tell them apart from the samples alone.
Example: Wagon-Wheel Effect
In film (24 fps), a wheel spinning at 25 revolutions per second appears to rotate at\( |25 - 24| = 1 \) revolution per second — slowly backwards. This is spatial aliasing: the frame rate is below the Nyquist rate for the spoke pattern.
Remark: Aliasing Is Irreversible
Once aliased components are mixed into the baseband, no amount of post-processing can separate them from legitimate low-frequency content. This is why anti-aliasing filtering must be applied before sampling (see Section 5.5).
Interactive Demo: Aliasing Demonstration
Click Run to execute the Python code
First run will download Python environment (~15MB)
5.4 — Ideal Reconstruction (Sinc Interpolation)
The sampling theorem is not merely an existence result — it provides an explicit reconstruction formula. If \( x(t) \) is band-limited to \( f_B < f_s/2 \), then the original signal is recovered exactly from its samples by sinc interpolation.
Theorem (Whittaker–Shannon Interpolation Formula)
For a band-limited signal sampled at \( f_s > 2f_B \):
\[ x(t) = \sum_{n=-\infty}^{\infty} x(nT_s)\,\text{sinc}\!\left(\frac{t - nT_s}{T_s}\right) \]
where \( \text{sinc}(u) = \frac{\sin(\pi u)}{\pi u} \) (with \( \text{sinc}(0) = 1 \)).
Derivation via Ideal Low-Pass Filter
To isolate the baseband replica from \( X_s(f) \), we apply the ideal low-pass filter:
\[ H(f) = T_s \cdot \text{rect}\!\left(\frac{f}{f_s}\right) = \begin{cases} T_s & |f| < f_s/2 \\ 0 & |f| > f_s/2 \end{cases} \]
The reconstructed spectrum is \( X(f) = H(f) \cdot X_s(f) \). Taking the inverse Fourier transform of \( H(f) \) gives \( h(t) = \text{sinc}(t/T_s) \). Since \( x_s(t) = \sum_n x(nT_s)\,\delta(t - nT_s) \), the convolution\( x(t) = h(t) * x_s(t) \) yields the Whittaker–Shannon formula directly.
Remark: Practical Limitations of Sinc Interpolation
The sinc function decays as \( 1/|t| \) and is non-causal (it extends to \( t = -\infty \)), so perfect reconstruction requires infinitely many samples and infinite delay. In practice, windowed-sinc filters (e.g., Lanczos) truncate the sum and taper the sinc to control the Gibbs phenomenon.
Interactive Demo: Sinc Interpolation (Perfect Reconstruction)
Click Run to execute the Python code
First run will download Python environment (~15MB)
5.5 — Practical ADC/DAC
Real analog-to-digital and digital-to-analog converters differ from the ideal sampling/reconstruction model in several important ways. A complete conversion pipeline typically includes four stages.
1. Anti-Aliasing Filter (before ADC)
An analog low-pass filter attenuates all frequency components above \( f_s/2 \)before sampling. Because real filters have a finite transition band (they cannot be brick-wall), engineers typically choose \( f_s \) somewhat above \( 2f_B \)to give the filter room to roll off. Common choices are Butterworth (maximally flat passband) and Chebyshev (steeper roll-off with passband ripple) filters of order 4–8.
2. Sample-and-Hold (S/H)
An ideal sampler produces a Dirac impulse, but a real ADC needs the voltage to remain constant during quantization. The sample-and-hold circuit acquires the signal at time \( nT_s \)and holds it steady for the duration of the conversion. The held waveform is a staircase approximation whose spectrum is shaped by the zero-order hold:
\[ H_{\text{ZOH}}(f) = T_s\,\text{sinc}(fT_s)\,e^{-j\pi f T_s} \]
3. Quantization
The held voltage is mapped to the nearest discrete level among \( 2^b \) levels (for a \( b \)-bit ADC). Quantization introduces quantization noise, which for a uniformly distributed input is approximately white with power:
\[ \sigma_q^2 = \frac{\Delta^2}{12}, \qquad \Delta = \frac{V_{\text{FSR}}}{2^b} \]
where \( V_{\text{FSR}} \) is the full-scale range and \( \Delta \) is the step size (LSB). The signal-to-quantization-noise ratio (SQNR) for a full-scale sinusoid is:
\[ \text{SQNR} \approx 6.02\,b + 1.76 \;\text{dB} \]
Example: 16-bit CD Audio SQNR
For \( b = 16 \) bits: SQNR \( \approx 6.02 \times 16 + 1.76 = 98.08 \) dB — more than sufficient to cover the roughly 90 dB dynamic range of orchestral music.
4. Reconstruction Filter (after DAC)
The DAC outputs a zero-order held staircase. A reconstruction (smoothing) low-pass filter removes the spectral images above \( f_s/2 \) and compensates for the sinc-shaped droop introduced by the ZOH. In high-fidelity audio, this is often an active analog filter followed by a buffer amplifier.
Remark: The Full ADC/DAC Chain
Anti-alias filter \( \rightarrow \) S/H \( \rightarrow \) Quantizer\( \rightarrow \) Digital Processing \( \rightarrow \) DAC\( \rightarrow \) Reconstruction filter. Each stage introduces imperfections (filter passband ripple, aperture jitter, quantization noise, DAC nonlinearity), but careful design keeps total distortion below perceptible thresholds.
5.6 — Oversampling and Its Benefits
Sampling well above the Nyquist rate — oversampling — provides surprising practical advantages even though the sampling theorem says \( f_s > 2f_B \) is sufficient.
Definition: Oversampling Ratio (OSR)
The oversampling ratio is \( \text{OSR} = \frac{f_s}{2f_B} \). An OSR of 1 corresponds to Nyquist-rate sampling; modern sigma-delta ADCs may use OSR = 64, 128, or higher.
Quantization Noise Spreading
Quantization noise has fixed total power \( \Delta^2/12 \) that is spread uniformly over \( [0, f_s/2] \). By oversampling, the noise power spectral density decreases as \( 1/f_s \). After digital low-pass filtering to the band of interest \( [0, f_B] \), only a fraction \( 1/\text{OSR} \) of the noise remains, yielding an effective resolution improvement of:
\[ \Delta b = \frac{1}{2}\log_2(\text{OSR}) \;\text{ extra bits} \]
Example: 4x Oversampling
With OSR = 4: \( \Delta b = \frac{1}{2}\log_2(4) = 1 \) extra bit of resolution. A 14-bit ADC running at \( 4 \times \) the Nyquist rate achieves roughly 15-bit effective resolution after decimation filtering.
Noise Shaping and Sigma-Delta ADCs
Sigma-delta (\( \Sigma\Delta \)) modulators go further: they use feedback toshape the quantization noise spectrum, pushing most of the noise energy to high frequencies (above \( f_B \)) where the decimation filter will remove it. A first-order noise shaper applies the transfer function:
\[ N(z) = (1 - z^{-1})\,E(z) \]
where \( E(z) \) is the quantization error. The magnitude response\( |1 - e^{-j\omega}| = 2|\sin(\omega/2)| \) is small at low frequencies and large at high frequencies, concentrating noise where it will be filtered away.
Theorem: Sigma-Delta Resolution Scaling
For an \( L \)-th order sigma-delta modulator with oversampling ratio OSR, the effective number of bits (ENOB) improvement over Nyquist-rate conversion is approximately:
\[ \Delta b \approx (L + 0.5)\log_2(\text{OSR}) - \frac{1}{2}\log_2\!\left(\frac{\pi^{2L}}{2L+1}\right) \]
Remark: Practical Sigma-Delta Converters
Modern audio ADCs (e.g., the AK5397 or PCM4222) use 3rd- or 4th-order sigma-delta modulators at 64x or 128x oversampling with only a 1-bit internal quantizer, yet achieve 24-bit effective resolution after decimation. This approach trades temporal resolution (very high sample rate) for amplitude resolution (very coarse quantizer), relying on noise shaping and digital filtering to recover dynamic range.
5.7 — Bandpass Sampling
The standard Nyquist theorem assumes the signal is baseband (frequencies from 0 to\( f_B \)). Many practical signals — radio-frequency carriers, ultrasound echoes, radar returns — are bandpass: their energy lies in a band\( [f_L, f_H] \) with bandwidth \( B = f_H - f_L \) centered far from zero. In such cases, we can sample at rates much lower than \( 2f_H \).
Theorem (Bandpass Sampling Theorem)
Let \( x(t) \) be a real bandpass signal with spectral support on\( [f_L, f_H] \) and bandwidth \( B = f_H - f_L \). Then\( x(t) \) can be perfectly reconstructed from uniform samples at rate\( f_s \) provided:
\[ f_s \geq 2B \]
and \( f_s \) is chosen so that the spectral replicas do not overlap. The permissible sampling rates are:
\[ \frac{2f_H}{n} \leq f_s \leq \frac{2f_L}{n-1}, \qquad n = 1, 2, \ldots, \left\lfloor \frac{f_H}{B} \right\rfloor \]
Example: Software-Defined Radio
An FM radio signal at carrier 100 MHz with 200 kHz bandwidth has\( f_L = 99.9 \) MHz, \( f_H = 100.1 \) MHz, \( B = 0.2 \) MHz.
Instead of sampling at \( 2 \times 100.1 = 200.2 \) MHz (as baseband Nyquist would require), bandpass sampling allows \( f_s \geq 2 \times 0.2 = 0.4 \) MHz. In practice one might choose \( f_s \approx 0.5 \) MHz, reducing the data rate by a factor of 400.
The sampling process naturally performs downconversion: the bandpass signal aliases into the baseband \( [0, f_s/2] \), centered at a convenient intermediate frequency.
Remark: Jitter Sensitivity
Bandpass sampling is far more sensitive to clock jitter than baseband sampling. A timing error \( \Delta t \) at carrier frequency \( f_c \) causes a phase error \( 2\pi f_c \Delta t \), which can be enormous when\( f_c \gg B \). High-quality crystal oscillators or PLLs are essential.
Interactive Demo: Spectral Replication Under Sampling
This visualization shows how sampling a continuous signal creates periodic copies of its spectrum in the frequency domain. When the sampling rate satisfies the Nyquist condition, the copies remain separated and the original can be recovered.
Click Run to execute the Python code
First run will download Python environment (~15MB)
Chapter Summary
Key Results
- Ideal sampling multiplies by a Dirac comb, creating spectral replicas at multiples of \( f_s \).
- The Shannon–Nyquist theorem: sample at \( f_s > 2f_B \) to preserve all information.
- Aliasing folds frequencies above \( f_s/2 \) back into the baseband, irreversibly.
- Perfect reconstruction uses sinc interpolation (Whittaker–Shannon formula).
Practical Takeaways
- Always apply an anti-aliasing filter before the ADC.
- Quantization noise SQNR \( \approx 6.02b + 1.76 \) dB.
- Oversampling gains \( \frac{1}{2}\log_2(\text{OSR}) \) extra bits; noise shaping gains much more.
- Bandpass signals need only \( f_s \geq 2B \), not \( 2f_H \).