Part V: From Code to Bitstream | Chapter 15

Timing Closure

Setup and hold, slack and the critical path, maximum clock frequency, pipelining, metastability and its MTBF, and clock domain crossing.

1. What “fast enough” means

In Chapter 13 the timing summary reported a number called WNS, and we said it must be positive. This chapter explains where that number comes from and what to do when it is negative. The idea is simple: a synchronous circuit works only if every signal launched by one clock edge has arrived and settled at its destination flip-flop before the next edge samples it — and has not changed too early either.

The timing analyser cuts the design into timing paths. Each path starts at a point where data is launched and ends where it is captured:

register → register: from a flip-flop's clock pin, through its output, the logic and routing, to the D input of another flip-flop. This is the vast majority of paths.
input → register: from a top-level input pin to the first flip-flop. Timing it needs to know when the outside world changes the input (set_input_delay).
register → output: from the last flip-flop to an output pin (set_output_delay).

For buttons, switches and LEDs the input/output timing does not matter — a human cannot press a button in 8 ns — so in this course those paths are left unconstrained and handled with synchronisers. We focus on register-to-register paths.

A register-to-register timing path. The clock reaches the launch and capture flip-flops through a clock tree with slightly different delays; the difference is the clock skew.

2. The setup condition

Take the clock edge at time 0 at the clock pin. Through the clock tree it reaches the launch flip-flop at$t_L$ and the capture flip-flop at $t_C$. The clock skew is$t_\text{skew} = t_C - t_L$. Three delays make up the data path:

$t_\text{cq}$ — clock-to-Q: after its clock edge, the launch flip-flop needs this long to show the new value at Q.
$t_\text{logic}$ — the delay through the LUTs and carry chains the signal passes (each one is a logic level).
$t_\text{route}$ — the delay of the programmable wires between them. On an FPGA this is often as large as the logic delay, or larger.

The capture flip-flop needs its D input stable for a setup time $t_\text{su}$before its clock edge. The data launched at edge 0 must be captured by the next edge, which reaches the capture flip-flop at $T + t_C$. So:

\[ \underbrace{t_L + t_\text{cq} + t_\text{logic} + t_\text{route}}_{\text{data arrival}} \;\le\; \underbrace{T + t_C - t_\text{su}}_{\text{data required}} \quad\Longleftrightarrow\quad t_\text{cq} + t_\text{logic} + t_\text{route} + t_\text{su} \;\le\; T + t_\text{skew} \]

The setup slack is how much room is left: required time minus arrival time. The real tools also subtract a clock uncertainty $t_\text{unc}$ for jitter, and they use the slowest (maximum) delays of the slow process corner for the data path.

\[ \text{slack}_\text{setup} = \big(T + t_\text{skew} - t_\text{su} - t_\text{unc}\big) - \big(t_\text{cq} + t_\text{logic} + t_\text{route}\big)_\text{max} \;\ge\; 0 \]

Setting the slack to zero and solving for $T$ gives the shortest period this path tolerates, and hence its maximum clock frequency:

\[ f_\text{max} = \frac{1}{T_\text{min}} = \frac{1}{t_\text{cq} + t_\text{logic} + t_\text{route} + t_\text{su} + t_\text{unc} - t_\text{skew}} \]

A worked example with illustrative numbers: $t_\text{cq} = 0.5$ ns, three logic levels totalling 0.9 ns, 2.4 ns of routing, $t_\text{su} = 0.1$ ns, skew −0.1 ns, no uncertainty. Arrival is 3.8 ns; required is $8 - 0.1 - 0.1 = 7.8$ ns; slack is +4.0 ns, and $f_\text{max} = 1/4.0\ \text{ns} = 250\ \text{MHz}$.

3. The hold condition

Setup says the data must not be too late. Hold says it must not be too early. A flip-flop needs its input to stay stable for a hold time $t_h$ after its clock edge. The danger is the same clock edge: at edge 0 the launch flip-flop starts sending a new value while the capture flip-flop is still sampling the old one. If the path is very short and the capture clock arrives late, the new value can overwrite the old one inside the hold window.

\[ t_L + t_\text{cq,min} + t_\text{comb,min} \;\ge\; t_C + t_h \quad\Longleftrightarrow\quad \text{slack}_\text{hold} = \big(t_\text{cq} + t_\text{comb}\big)_\text{min} - \big(t_h + t_\text{skew}\big) \;\ge\; 0 \]

Here $t_\text{comb} = t_\text{logic} + t_\text{route}$, using the fastest (minimum) delays. Two observations follow directly from the two inequalities:

The period $T$ does not appear in the hold condition. A hold violation cannot be fixed by slowing the clock.
Positive skew (capture clock late) helps setup but hurts hold, by exactly the same amount. That is why FPGAs distribute clocks on dedicated low-skew networks, and why you must never clock logic from ordinary signals (Chapter 14).

Setup and hold seen at the capture flip-flop. The data launched at edge 0 must change only after the hold window of the capture edge at t_C (hold slack) and must be settled before the setup window of the next capture edge at T + t_C (setup slack).

Key idea: setup is about the next edge, hold about the same edge

Setup compares the slowest arrival with the next capture edge, so it depends on the clock period. Hold compares the fastest arrival with the same capture edge, so it does not. Vivado's router fixes almost all hold violations by itself, by adding wire delay to short paths; your job is mostly setup.

4. Slack, WNS, TNS and the critical path

The analyser computes the slack of every path, for both setup and hold. The path with the smallest setup slack is the critical path: it alone determines $f_\text{max}$. The summary numbers you met in Chapter 13 are aggregates over all endpoints:

\[ \text{WNS} = \min_\text{paths} \text{slack}_\text{setup}, \qquad \text{TNS} = \sum_\text{endpoints} \min\big(0,\ \text{worst slack at that endpoint}\big), \qquad \text{WHS} = \min_\text{paths} \text{slack}_\text{hold} \]

WNS tells you how far you are from closing timing; TNS tells you how widespread the problem is. A WNS of −0.3 ns with TNS −0.3 ns is one bad path; the same WNS with TNS −250 ns is a design-wide problem. Here is the shape of one path in a setup timing report (abridged; the numbers are illustrative but consistent):

report_timing (abridged, illustrative)

Slack (MET) :             1.234ns  (required time - arrival time)
  Source:                 acc_reg[3]/C   (rising edge-triggered cell FDRE clocked by sys_clk)
  Destination:            sum_reg[17]/D  (rising edge-triggered cell FDRE clocked by sys_clk)
  Path Group:             sys_clk
  Path Type:              Setup (Max at Slow Process Corner)
  Requirement:            8.000ns  (sys_clk rise@8.000ns - sys_clk rise@0.000ns)
  Data Path Delay:        6.650ns  (logic 3.410ns (51.3%)  route 3.240ns (48.7%))
  Logic Levels:           9  (CARRY4=6 LUT2=1 LUT4=2)
  Clock Path Skew:        -0.045ns
  Clock Uncertainty:      0.035ns
  ...
  required time           7.884
  arrival time           -6.650
  -----------------------------
  slack                   1.234

Line	What it tells you
Source / Destination	the launch and capture flip-flops: find them in your HDL
Requirement	the time allowed, normally one period (8 ns)
Data Path Delay	t_cq + logic + routing; the logic/route split hints at the fix
Logic Levels	how many LUTs/carry blocks in series: the main lever you control
Clock Path Skew	t_skew; small on a global clock, large means a clocking problem
required − arrival	T + skew − t_su − uncertainty, minus the data path delay

Check the arithmetic: required $= 8.000 - 0.045 - 0.035 - t_\text{su}$ with$t_\text{su} \approx 0.036$ ns gives 7.884 ns, and $7.884 - 6.650 = 1.234$ ns. If routing dominates the delay, the cells are far apart (high fanout, congestion, a resource in a fixed column); if logic dominates, there are too many levels between registers.

5. Pipelining: the main tool for setup

When a path has too much logic for one clock period, cut it into pieces with registers in between. Each piece then has a full period to itself. This is pipelining. The price is latency: a result appears more cycles after its inputs. The gain is throughput: a new input can still enter every cycle, at a higher clock rate.

Our example computes a four-element dot product of 8-bit unsigned numbers,$y = a_0b_0 + a_1b_1 + a_2b_2 + a_3b_3$, packed into two 32-bit inputs. The result needs 18 bits: each product is at most $255^2 = 65\,025$, and four of them sum to at most$260\,100 < 2^{18}$. The first version registers the inputs and the output, and puts four multipliers and three adders between them — one long register-to-register path.

dot4 — one long path (latency 2)

Verilog

module dot4 (
    input  wire        clk,
    input  wire [31:0] a,        // four 8-bit unsigned lanes a[7:0] .. a[31:24]
    input  wire [31:0] b,
    output reg  [17:0] y = 0     // a0*b0 + a1*b1 + a2*b2 + a3*b3, latency 2
);
    reg [31:0] a_r = 0, b_r = 0;
    always @(posedge clk) begin
        a_r <= a;
        b_r <= b;
        // One long register-to-register path: 4 multipliers and 3 adders
        y <= a_r[ 7: 0] * b_r[ 7: 0] + a_r[15: 8] * b_r[15: 8]
           + a_r[23:16] * b_r[23:16] + a_r[31:24] * b_r[31:24];
    end
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity dot4 is
    port (
        clk : in  std_logic;
        a   : in  std_logic_vector(31 downto 0);  -- four 8-bit unsigned lanes
        b   : in  std_logic_vector(31 downto 0);
        y   : out std_logic_vector(17 downto 0)   -- sum of products, latency 2
    );
end entity;

architecture rtl of dot4 is
    signal a_r, b_r : unsigned(31 downto 0) := (others => '0');
    signal y_r      : unsigned(17 downto 0) := (others => '0');
begin
    process (clk)
    begin
        if rising_edge(clk) then
            a_r <= unsigned(a);
            b_r <= unsigned(b);
            -- One long register-to-register path: 4 multipliers and 3 adders
            y_r <= resize(a_r( 7 downto  0) * b_r( 7 downto  0), 18)
                 + resize(a_r(15 downto  8) * b_r(15 downto  8), 18)
                 + resize(a_r(23 downto 16) * b_r(23 downto 16), 18)
                 + resize(a_r(31 downto 24) * b_r(31 downto 24), 18);
        end if;
    end process;
    y <= std_logic_vector(y_r);
end architecture;

The pipelined version computes exactly the same thing, but stores the four products, then the two pair sums, then the total, each in its own register. The longest path is now one multiplier or one adder.

dot4_pipe — the same sum in four stages (latency 4)

Verilog

module dot4_pipe (
    input  wire        clk,
    input  wire [31:0] a,        // four 8-bit unsigned lanes
    input  wire [31:0] b,
    output reg  [17:0] y = 0     // same sum of products, latency 4
);
    reg [31:0] a_r = 0, b_r = 0;                     // stage 1: inputs
    reg [15:0] p0 = 0, p1 = 0, p2 = 0, p3 = 0;       // stage 2: products
    reg [16:0] s01 = 0, s23 = 0;                     // stage 3: pair sums
    always @(posedge clk) begin
        a_r <= a;
        b_r <= b;
        p0  <= a_r[ 7: 0] * b_r[ 7: 0];
        p1  <= a_r[15: 8] * b_r[15: 8];
        p2  <= a_r[23:16] * b_r[23:16];
        p3  <= a_r[31:24] * b_r[31:24];
        s01 <= p0 + p1;
        s23 <= p2 + p3;
        y   <= s01 + s23;                            // stage 4: output
    end
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity dot4_pipe is
    port (
        clk : in  std_logic;
        a   : in  std_logic_vector(31 downto 0);  -- four 8-bit unsigned lanes
        b   : in  std_logic_vector(31 downto 0);
        y   : out std_logic_vector(17 downto 0)   -- same sum of products, latency 4
    );
end entity;

architecture rtl of dot4_pipe is
    signal a_r, b_r       : unsigned(31 downto 0) := (others => '0');  -- stage 1
    signal p0, p1, p2, p3 : unsigned(15 downto 0) := (others => '0');  -- stage 2
    signal s01, s23       : unsigned(16 downto 0) := (others => '0');  -- stage 3
    signal y_r            : unsigned(17 downto 0) := (others => '0');  -- stage 4
begin
    process (clk)
    begin
        if rising_edge(clk) then
            a_r <= unsigned(a);
            b_r <= unsigned(b);
            p0  <= a_r( 7 downto  0) * b_r( 7 downto  0);
            p1  <= a_r(15 downto  8) * b_r(15 downto  8);
            p2  <= a_r(23 downto 16) * b_r(23 downto 16);
            p3  <= a_r(31 downto 24) * b_r(31 downto 24);
            s01 <= resize(p0, 17) + p1;
            s23 <= resize(p2, 17) + p3;
            y_r <= resize(s01, 18) + s23;
        end if;
    end process;
    y <= std_logic_vector(y_r);
end architecture;

Where the registers sit. dot4 has one combinational path from a_r/b_r to y; dot4_pipe cuts it after the multipliers and after the first adders. Each cut adds one cycle of latency.

The testbench drives both designs with the same pseudo-random inputs from a 32-bit LFSR, keeps a short history of the correct results, and checks that y of dot4 equals the result from two edges ago and y of dot4_pipe the result from four edges ago — for 196 consecutive cycles. Both languages print the same line.

Testbench: same results, two extra cycles of latency

Verilog

`timescale 1ns/1ps
module dot4_tb;
    reg         clk = 0;
    reg  [31:0] a = 0, b = 0;
    wire [17:0] y2, y4;
    reg  [31:0] lfsr = 32'hACE1_2468;
    reg  [17:0] hist [0:4];            // hist[k]: correct result for inputs k edges ago
    integer i, k, errors = 0, checks = 0;

    dot4      u_flat (.clk(clk), .a(a), .b(b), .y(y2));
    dot4_pipe u_pipe (.clk(clk), .a(a), .b(b), .y(y4));

    always #4 clk = ~clk;

    // Reference model of the arithmetic
    function [17:0] ref_dot(input [31:0] x, input [31:0] z);
        ref_dot = x[7:0] * z[7:0] + x[15:8] * z[15:8]
                + x[23:16] * z[23:16] + x[31:24] * z[31:24];
    endfunction

    function [31:0] step(input [31:0] s);   // 32-bit Galois LFSR
        step = {s[30:0], 1'b0} ^ (s[31] ? 32'h04C1_1DB7 : 32'h0);
    endfunction

    initial begin
        for (k = 0; k <= 4; k = k + 1) hist[k] = 0;
        for (i = 0; i < 200; i = i + 1) begin
            @(negedge clk);
            // one clock edge has passed: age the history by one
            for (k = 4; k > 0; k = k - 1) hist[k] = hist[k-1];
            // y2 must equal the result for inputs applied 2 edges ago, y4 for 4
            if (i >= 4) begin
                checks = checks + 1;
                if (y2 !== hist[2] || y4 !== hist[4]) errors = errors + 1;
            end
            // apply new random inputs
            lfsr = step(lfsr); a = lfsr;
            lfsr = step(lfsr); b = lfsr;
            hist[0] = ref_dot(a, b);
        end
        if (errors == 0)
            $display("PASS: %0d checks, flat latency 2, pipelined latency 4, last y = %0d", checks, y4);
        else
            $display("FAIL: %0d errors in %0d checks", errors, checks);
        $finish;
    end
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity dot4_tb is
end entity;

architecture sim of dot4_tb is
    signal clk    : std_logic := '0';
    signal a, b   : std_logic_vector(31 downto 0) := (others => '0');
    signal y2, y4 : std_logic_vector(17 downto 0);
    signal done   : boolean := false;

    -- Reference model of the arithmetic
    function ref_dot(x, z : std_logic_vector(31 downto 0)) return unsigned is
        variable s : unsigned(17 downto 0) := (others => '0');
    begin
        for k in 0 to 3 loop
            s := s + resize(unsigned(x(8*k + 7 downto 8*k)) * unsigned(z(8*k + 7 downto 8*k)), 18);
        end loop;
        return s;
    end function;

    function step(s : unsigned(31 downto 0)) return unsigned is   -- 32-bit Galois LFSR
    begin
        if s(31) = '1' then
            return (s(30 downto 0) & '0') xor x"04C11DB7";
        else
            return s(30 downto 0) & '0';
        end if;
    end function;
begin
    u_flat : entity work.dot4      port map (clk => clk, a => a, b => b, y => y2);
    u_pipe : entity work.dot4_pipe port map (clk => clk, a => a, b => b, y => y4);

    clk <= not clk after 4 ns when not done;

    stimulus : process
        type hist_t is array (0 to 4) of unsigned(17 downto 0);
        variable hist   : hist_t := (others => (others => '0'));  -- hist(k): result for inputs k edges ago
        variable lfsr   : unsigned(31 downto 0) := x"ACE12468";
        variable va, vb : std_logic_vector(31 downto 0);
        variable errors, checks : natural := 0;
    begin
        for i in 0 to 199 loop
            wait until falling_edge(clk);
            -- one clock edge has passed: age the history by one
            for k in 4 downto 1 loop
                hist(k) := hist(k - 1);
            end loop;
            -- y2 must equal the result for inputs applied 2 edges ago, y4 for 4
            if i >= 4 then
                checks := checks + 1;
                if unsigned(y2) /= hist(2) or unsigned(y4) /= hist(4) then
                    errors := errors + 1;
                end if;
            end if;
            -- apply new random inputs
            lfsr := step(lfsr); va := std_logic_vector(lfsr);
            lfsr := step(lfsr); vb := std_logic_vector(lfsr);
            a <= va;
            b <= vb;
            hist(0) := ref_dot(va, vb);
        end loop;
        if errors = 0 then
            report "PASS: " & integer'image(checks) & " checks, flat latency 2, pipelined latency 4, last y = "
                & integer'image(to_integer(unsigned(y4)));
        else
            report "FAIL: " & integer'image(errors) & " errors in " & integer'image(checks) & " checks"
                severity error;
        end if;
        done <= true;
        wait;
    end process;
end architecture;

Run it — both print: PASS: 196 checks, flat latency 2, pipelined latency 4, last y = 42024

iverilog -g2012 -o tb dot4.v dot4_pipe.v dot4_tb.v && vvp tb
ghdl -a --std=08 dot4.vhd dot4_pipe.vhd dot4_tb.vhd
ghdl -e --std=08 dot4_tb && ghdl -r --std=08 dot4_tb

On the Zynq the multipliers map to DSP48E1 slices, which have optional internal registers before and after the multiplier. Synthesis can absorb the pipeline registers you write into those built-in registers, so pipelining a multiplier often costs no extra fabric flip-flops at all — and a DSP slice reaches its rated speed only when those registers are used.

Common mistake: pipelining one branch only

If one signal goes through an extra register and another one it is combined with does not, the two arrive from different input samples and the result is silently wrong. Every path from the inputs to a given adder must cross the same number of registers. Count stages along every branch — or let a testbench like the one above do it for you.

6. Other ways to close timing

Retiming. Moving existing registers across logic so that stages become more balanced, without changing the latency. If stage 1 has 7 ns of logic and stage 2 has 2 ns, moving part of the logic from 1 to 2 helps. Vivado can do some of this automatically (a synthesis retiming option), but writing balanced stages yourself is more predictable.
Fewer logic levels. A 6-input LUT implements any function of six inputs in one level, so restructuring often helps: a wide comparison as a tree instead of a chain, a case instead of a long if/elsif priority chain when priority is not needed, one-hot state encoding (Chapter 12) instead of binary.
Less routing delay. A signal that drives hundreds of loads (a high fanout enable or reset) is slow because it must reach all of them. Registering it and duplicating the register helps; so does avoiding resets on datapath registers that do not need them.
Timing exceptions, with great care. set_false_path tells the analyser not to time a path at all; set_multicycle_path allows a path several periods (for example when a register is only enabled every fourth cycle). Both are promises you make. If the promise is false, the tool happily reports success and the hardware fails. Use them only for paths you can justify in writing, and prefer fixing the design.

On the board: measuring the effect of pipelining in Vivado

dot4 has 83 input/output bits — far more than the board's switches, buttons and LEDs. Build it out of context (as a block that will later sit inside a larger design), which needs no pin constraints. Run this once with dot4 and once with dot4_pipe:

ooc.tcl — vivado -mode batch -source ooc.tcl

set top dot4            ;# then: dot4_pipe
read_verilog $top.v
synth_design -top $top -part xc7z020clg400-1 -mode out_of_context
create_clock -name clk -period 4.000 [get_ports clk]   ;# ask for 250 MHz
opt_design
place_design
route_design
report_timing_summary -file $top.timing.rpt
report_utilization    -file $top.util.rpt

Compare WNS for the two designs. Then shorten the period step by step until each one fails; estimate each $f_\text{max}$ as $1/(T - \text{WNS})$.
Open the utilisation reports: how many DSP48E1 and flip-flops does each version use? Were your pipeline registers absorbed into the DSPs?
In dot4.timing.rpt, find the critical path: how many logic levels, and what share is routing?

7. Metastability and its MTBF

Everything so far assumed the data meets setup and hold. A signal from a button, or from logic on another clock, has no fixed relation to our clock, so sooner or later it will change inside the setup/hold window. The flip-flop is then metastable: its internal latch is balanced between 0 and 1, like a ball on top of a hill, and its output may sit at an invalid level or oscillate for a while before falling one way. Which way, and when, is random. The probability that it is still undecided after a time$t$ decays exponentially, $\propto e^{-t/\tau}$, where $\tau$ is a time constant of the flip-flop, typically tens of picoseconds in modern silicon.

A synchroniser gives the flip-flop time. In a two-flip-flop synchroniser the first stage samples the asynchronous signal; the second samples the first one a full period later. The design fails only if the first stage is still unresolved after the available resolution time$t_r$. The standard model of the mean time between such failures is

\[ \text{MTBF} = \frac{e^{\,t_r/\tau}}{T_0 \, f_\text{clk} \, f_\text{data}}, \qquad t_r \approx (N-1)\big(T - t_\text{cq} - t_\text{su} - t_\text{route}\big) + \text{slack of the path that uses the result} \]

$f_\text{clk}$ is the sampling clock, $f_\text{data}$ the rate at which the asynchronous input changes, $T_0$ a constant related to the width of the dangerous window, and$N$ the number of flip-flops in the chain. The denominator counts how often you risk a metastable event; the numerator says how unlikely each one is to last long enough to matter. Because$t_r$ sits in an exponent, every extra stage multiplies the MTBF by a gigantic factor.

The values of $\tau$ and $T_0$ below are illustrative, chosen to be of a plausible order of magnitude; real values depend on the silicon and the conditions. The shape of the result is what matters: one flip-flop fails in a fraction of a millisecond, two are vastly more than enough at 125 MHz, and at high clock rates the margin shrinks quickly.

Python

script.py58 lines

import numpy as np
import matplotlib.pyplot as plt

# MTBF of an N-flip-flop synchroniser:  MTBF = exp(t_r / tau) / (T0 * f_clk * f_data)
# tau and T0 are ILLUSTRATIVE values of a plausible order of magnitude; the
# real ones depend on the silicon and are not published per flip-flop.
TAU   = 0.12e-9    # metastability time constant (s)
T0    = 0.5e-9     # metastability window parameter (s)
T_OH  = 0.6e-9     # clock-to-q + setup + routing between stages (s)
T_S   = 0.5e-9     # slack left in the path that finally uses the value (s)
F_DATA = 10e6      # the asynchronous input changes 10 million times per second

YEAR = 365.25 * 24 * 3600

def t_resolve(n, f_clk):
    """Time a metastable first stage has to settle with n flip-flops in the chain."""
    T = 1.0 / f_clk
    return (n - 1) * (T - T_OH) + T_S

def log10_mtbf(n, f_clk):
    # work with logarithms: exp(t_r/tau) overflows a float for long t_r
    return t_resolve(n, f_clk) / TAU / np.log(10) - np.log10(T0 * f_clk * F_DATA)

def describe(lg):
    s = lg
    if s < np.log10(60):        return f"{10**s:.3g} s"
    if s < np.log10(YEAR):      return f"{10**s / 86400:.3g} days"
    return f"10^{s - np.log10(YEAR):.0f} years"

print(f"tau = {TAU*1e12:.0f} ps, T0 = {T0*1e9:.1f} ns, f_data = {F_DATA/1e6:.0f} MHz (illustrative)")
print(f"{'f_clk':>9} | {'1 FF':>14} | {'2 FF':>14} | {'3 FF':>14}")
for f in (125e6, 250e6, 400e6, 500e6):
    row = [describe(log10_mtbf(n, f)) for n in (1, 2, 3)]
    print(f"{f/1e6:>5.0f} MHz | " + " | ".join(f"{r:>14}" for r in row))

f = np.linspace(50e6, 600e6, 400)
plt.style.use('dark_background')
fig, ax = plt.subplots(figsize=(10, 5.2))
fig.patch.set_facecolor('#0a0a0a'); ax.set_facecolor('#0f0f0f')
for n, col in ((1, '#f87171'), (2, '#fb923c'), (3, '#38bdf8')):
    ax.plot(f / 1e6, log10_mtbf(n, f), color=col, lw=2,
            label=f"{n} flip-flop{'s' if n > 1 else ''}")
refs = [(0, '1 second'), (np.log10(86400), '1 day'), (np.log10(YEAR), '1 year'),
        (np.log10(1.4e10 * YEAR), 'age of the universe')]
for lg, name in refs:
    ax.axhline(lg, color='#4b5563', ls=':', lw=1)
    ax.text(598, lg + 0.6, name, color='#9ca3af', fontsize=9, ha='right')
ax.axvline(125, color='#fbbf24', ls='--', lw=1)
ax.text(130, 42, 'PYNQ-Z2\n125 MHz', color='#fbbf24', fontsize=9, va='top')
ax.set_ylim(-8, 45); ax.set_xlim(50, 600)
ax.set_xlabel('destination clock frequency (MHz)')
ax.set_ylabel('log$_{10}$ MTBF (seconds)')
ax.set_title('Synchroniser MTBF vs clock frequency (illustrative τ = 120 ps, T$_0$ = 0.5 ns, f$_{data}$ = 10 MHz)',
             color='#fdba74', fontsize=11)
ax.legend(loc='upper right', bbox_to_anchor=(1.0, 0.86), facecolor='#111827', edgecolor='#374151')
for sp in ax.spines.values(): sp.set_edgecolor('#374151')
plt.tight_layout()
plt.savefig('output.png', dpi=120, facecolor='#0a0a0a')

Click Run to execute the Python code

Code will be executed with Python 3 on the server

Key idea: you cannot eliminate metastability, only make it absurdly rare

No circuit can sample an asynchronous signal with zero failure probability. A synchroniser trades latency (one clock period per stage) for an exponential improvement in MTBF. Two stages at 125 MHz put the expected failure far beyond the lifetime of the hardware; that is why every asynchronous input in this course goes through one.

8. Crossing clock domains

A design with two unrelated clocks — say a 125 MHz system clock and a clock recovered from a video input — has two clock domains. A signal going from one to the other is a clock domain crossing (CDC), and to the receiving domain it is just as asynchronous as a button. Three cases, three techniques:

What crosses	Technique	Why
a slow single-bit level	2-FF synchroniser	the level lasts many destination cycles
a single-cycle pulse	toggle + 2-FF + edge detect	a short pulse could fall between edges
a counter / pointer	Gray code + 2-FF per bit	only one bit changes per step
a multi-bit data word	handshake or async FIFO (Ch. 17)	bits may resolve on different cycles

The 2-FF synchroniser is the building block. The ASYNC_REG attribute tells Vivado that these flip-flops form a synchroniser: they are kept (not merged or optimised), placed in the same slice to minimise the wire between them, and recognised as such by the CDC reports.

sync_2ff — two flip-flops for one asynchronous bit

Verilog

module sync_2ff (
    input  wire clk,       // destination clock
    input  wire d,         // asynchronous level (from another clock domain or a pin)
    output wire q          // synchronised copy, 2 clk cycles later
);
    // ASYNC_REG tells Vivado: keep these two flip-flops, place them side by
    // side, and treat the first one as a synchroniser stage.
    (* ASYNC_REG = "TRUE" *) reg s0 = 1'b0;
    (* ASYNC_REG = "TRUE" *) reg s1 = 1'b0;
    always @(posedge clk) begin
        s0 <= d;           // may go metastable
        s1 <= s0;          // has a full period for s0 to settle
    end
    assign q = s1;
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;

entity sync_2ff is
    port (
        clk : in  std_logic;   -- destination clock
        d   : in  std_logic;   -- asynchronous level (from another clock domain or a pin)
        q   : out std_logic    -- synchronised copy, 2 clk cycles later
    );
end entity;

architecture rtl of sync_2ff is
    signal s0, s1 : std_logic := '0';
    -- ASYNC_REG tells Vivado: keep these two flip-flops, place them side by
    -- side, and treat the first one as a synchroniser stage.
    attribute ASYNC_REG : string;
    attribute ASYNC_REG of s0 : signal is "TRUE";
    attribute ASYNC_REG of s1 : signal is "TRUE";
begin
    process (clk)
    begin
        if rising_edge(clk) then
            s0 <= d;           -- may go metastable
            s1 <= s0;          -- has a full period for s0 to settle
        end if;
    end process;
    q <= s1;
end architecture;

A one-cycle pulse in the source domain cannot go through a 2-FF synchroniser directly: if the destination clock is slower, its edges may miss the pulse entirely. The toggle pulse synchroniser turns each pulse into a change of level of a toggle flip-flop. A level stays put until the next pulse, so it cannot be missed; the destination synchronises it and detects each change with an XOR, producing exactly one destination-clock pulse per source pulse. The condition is that source pulses are spaced by at least a few destination cycles — otherwise two toggles could cancel before they are seen. When the source needs to know that the pulse arrived (or must send faster), a handshake sends an acknowledge back through a second synchroniser.

pulse_sync — one pulse in, one pulse out, across unrelated clocks

Verilog

module pulse_sync (
    input  wire clk_a,     // source clock
    input  wire pulse_a,   // one-cycle pulse in the clk_a domain
    input  wire clk_b,     // destination clock, unrelated to clk_a
    output wire pulse_b    // one-cycle pulse in the clk_b domain
);
    // Source side: turn each pulse into a change of level
    reg toggle_a = 1'b0;
    always @(posedge clk_a)
        if (pulse_a) toggle_a <= ~toggle_a;

    // Destination side: 2-FF synchroniser plus one more FF to see the change
    (* ASYNC_REG = "TRUE" *) reg s0 = 1'b0;
    (* ASYNC_REG = "TRUE" *) reg s1 = 1'b0;
    reg s2 = 1'b0;
    always @(posedge clk_b) begin
        s0 <= toggle_a;
        s1 <= s0;
        s2 <= s1;
    end
    assign pulse_b = s1 ^ s2;   // high for one clk_b cycle per level change
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;

entity pulse_sync is
    port (
        clk_a   : in  std_logic;   -- source clock
        pulse_a : in  std_logic;   -- one-cycle pulse in the clk_a domain
        clk_b   : in  std_logic;   -- destination clock, unrelated to clk_a
        pulse_b : out std_logic    -- one-cycle pulse in the clk_b domain
    );
end entity;

architecture rtl of pulse_sync is
    signal toggle_a   : std_logic := '0';
    signal s0, s1, s2 : std_logic := '0';
    attribute ASYNC_REG : string;
    attribute ASYNC_REG of s0 : signal is "TRUE";
    attribute ASYNC_REG of s1 : signal is "TRUE";
begin
    -- Source side: turn each pulse into a change of level
    process (clk_a)
    begin
        if rising_edge(clk_a) then
            if pulse_a = '1' then
                toggle_a <= not toggle_a;
            end if;
        end if;
    end process;

    -- Destination side: 2-FF synchroniser plus one more FF to see the change
    process (clk_b)
    begin
        if rising_edge(clk_b) then
            s0 <= toggle_a;
            s1 <= s0;
            s2 <= s1;
        end if;
    end process;
    pulse_b <= s1 xor s2;   -- high for one clk_b cycle per level change
end architecture;

The testbench runs two unrelated clocks: 8 ns and 10.3 ns, with an odd phase offset so that the relative phase of their edges drifts through every value during the run. It checks that the level copy follows within three destination cycles, sends 25 single pulses spaced 6 to 10 source cycles apart, and checks that exactly 25 one-cycle pulses arrive. Both simulators print PASS: level ok, 25 pulses sent, 25 received at 2025.65 ns. (A simulator never goes metastable — it resolves every sample cleanly — so this checks the logic, not the MTBF.)

Testbench with two unrelated clocks

Verilog

`timescale 1ns/1ps
module cdc_tb;
    // Two unrelated clocks: 125 MHz and about 97 MHz with an odd phase offset
    reg clk_a = 0, clk_b = 0;
    always #4 clk_a = ~clk_a;                    // 8 ns
    initial begin
        #1.7;
        forever #5.15 clk_b = ~clk_b;            // 10.3 ns
    end

    reg  lvl_a = 0, pulse_a = 0;
    wire lvl_b, pulse_b;
    integer i, sent = 0, received = 0, width_err = 0, lvl_err = 0;
    reg prev_b = 0;

    sync_2ff   u_lvl (.clk(clk_b), .d(lvl_a), .q(lvl_b));
    pulse_sync u_pls (.clk_a(clk_a), .pulse_a(pulse_a), .clk_b(clk_b), .pulse_b(pulse_b));

    // Count pulses in the destination domain; no pulse may last 2 cycles
    always @(posedge clk_b) begin
        if (pulse_b) received = received + 1;
        if (pulse_b && prev_b) width_err = width_err + 1;
        prev_b = pulse_b;
    end

    initial begin
        // 1. Level crossing: after 3 clk_b edges the copy must match
        for (i = 0; i < 6; i = i + 1) begin
            @(posedge clk_a) lvl_a <= ~lvl_a;
            repeat (3) @(posedge clk_b);
            #1 if (lvl_b !== lvl_a) lvl_err = lvl_err + 1;
        end
        // 2. Pulse crossing: 25 single-cycle pulses, 6 to 10 clk_a cycles apart
        for (i = 0; i < 25; i = i + 1) begin
            @(posedge clk_a) pulse_a <= 1'b1;
            @(posedge clk_a) pulse_a <= 1'b0;
            sent = sent + 1;
            repeat (5 + i % 5) @(posedge clk_a);
        end
        repeat (5) @(posedge clk_b);
        if (lvl_err == 0 && received == sent && width_err == 0)
            $display("PASS: level ok, %0d pulses sent, %0d received", sent, received);
        else
            $display("FAIL: lvl_err=%0d sent=%0d received=%0d width_err=%0d",
                     lvl_err, sent, received, width_err);
        $finish;
    end
endmodule

VHDL

library ieee;
use ieee.std_logic_1164.all;

entity cdc_tb is
end entity;

architecture sim of cdc_tb is
    -- Two unrelated clocks: 125 MHz and about 97 MHz with an odd phase offset
    signal clk_a, clk_b     : std_logic := '0';
    signal lvl_a, pulse_a   : std_logic := '0';
    signal lvl_b, pulse_b   : std_logic;
    signal received         : natural := 0;
    signal width_err        : natural := 0;
    signal done             : boolean := false;
begin
    clk_a <= not clk_a after 4 ns when not done;                 -- 8 ns
    clk_b_gen : process
    begin
        wait for 1.7 ns;
        while not done loop
            wait for 5.15 ns;
            clk_b <= not clk_b;                                  -- 10.3 ns
        end loop;
        wait;
    end process;

    u_lvl : entity work.sync_2ff   port map (clk => clk_b, d => lvl_a, q => lvl_b);
    u_pls : entity work.pulse_sync port map (clk_a => clk_a, pulse_a => pulse_a,
                                             clk_b => clk_b, pulse_b => pulse_b);

    -- Count pulses in the destination domain; no pulse may last 2 cycles
    counter : process (clk_b)
        variable prev_b : std_logic := '0';
    begin
        if rising_edge(clk_b) then
            if pulse_b = '1' then
                received <= received + 1;
                if prev_b = '1' then
                    width_err <= width_err + 1;
                end if;
            end if;
            prev_b := pulse_b;
        end if;
    end process;

    stimulus : process
        variable sent, lvl_err : natural := 0;
    begin
        -- 1. Level crossing: after 3 clk_b edges the copy must match
        for i in 0 to 5 loop
            wait until rising_edge(clk_a);
            lvl_a <= not lvl_a;
            for k in 1 to 3 loop
                wait until rising_edge(clk_b);
            end loop;
            wait for 1 ns;
            if lvl_b /= lvl_a then lvl_err := lvl_err + 1; end if;
        end loop;
        -- 2. Pulse crossing: 25 single-cycle pulses, 6 to 10 clk_a cycles apart
        for i in 0 to 24 loop
            wait until rising_edge(clk_a);
            pulse_a <= '1';
            wait until rising_edge(clk_a);
            pulse_a <= '0';
            sent := sent + 1;
            for k in 1 to 5 + i mod 5 loop
                wait until rising_edge(clk_a);
            end loop;
        end loop;
        for k in 1 to 5 loop
            wait until rising_edge(clk_b);
        end loop;
        if lvl_err = 0 and received = sent and width_err = 0 then
            report "PASS: level ok, " & integer'image(sent) & " pulses sent, "
                & integer'image(received) & " received";
        else
            report "FAIL: lvl_err=" & integer'image(lvl_err) & " sent=" & integer'image(sent)
                & " received=" & integer'image(received) & " width_err=" & integer'image(width_err)
                severity error;
        end if;
        done <= true;
        wait;
    end process;
end architecture;

Finally, tell the timing analyser about the crossing. Without a constraint, Vivado assumes the two clocks are related and tries to time the path from toggle_a to s0 against the (meaningless) worst-case distance between their edges, producing a spurious failure. Declaring the clocks asynchronous removes those paths from analysis — which is correct only because every crossing goes through a synchroniser:

cdc.xdc

## Two clocks from two pins that have no fixed phase relationship
create_clock -name clk_a -period 8.000  [get_ports clk_a]
create_clock -name clk_b -period 10.300 [get_ports clk_b]

## Tell the timing analyser not to time paths between them.  The
## synchronisers (ASYNC_REG) are what make those crossings safe.
set_clock_groups -asynchronous -group [get_clocks clk_a] -group [get_clocks clk_b]

Common mistake: a separate synchroniser for each bit of a bus

Eight 2-FF synchronisers on an 8-bit value do not synchronise the value. When several bits change at once, each one may resolve on a different destination cycle, so for one cycle the destination can see a mixture of old and new bits — a number that never existed. Use Gray code when the value steps by one, or a handshake or an asynchronous FIFO for arbitrary data (Chapter 17).

Practice Problems

Progress: 0/5 (0%)

⭐ Easy

A register-to-register path has $t_\text{cq} = 0.45$ ns, logic delay 2.1 ns, routing delay 3.0 ns, $t_\text{su} = 0.05$ ns and clock skew $+0.1$ ns (the capture clock arrives later). Ignoring uncertainty, find the setup slack at 125 MHz and the path's $f_\text{max}$.

⭐ Easy

Your design fails hold timing (WHS = −0.08 ns). A colleague suggests halving the clock frequency. Will that help? Explain with the hold inequality.

⭐⭐ Medium

A combinational path has 9.6 ns of logic plus routing; each register adds 0.6 ns of overhead ($t_\text{cq} + t_\text{su}$). (a) What is $f_\text{max}$ unpipelined? (b) You split it into three balanced stages. What are the new $f_\text{max}$, the latency in cycles and in nanoseconds, and the throughput?

⭐⭐ Medium

With the illustrative values of the MTBF program ($\tau = 120$ ps, register overhead 0.6 ns), by what factor does adding a third flip-flop to a synchroniser clocked at 125 MHz multiply the MTBF? Why is the answer so large?

⭐⭐⭐ Hard

A designer passes a 4-bit counter from clock domain A to domain B through four independent 2-FF synchronisers. (a) Give a concrete transition that can produce a value in B that the counter never had. (b) Why does sending the counter in Gray code fix it? (c) What does a Gray-coded pointer still not guarantee?

Problem Set Statistics

Total Problems

Attempted

Completion

Share:X Reddit LinkedIn

← 14. First Lights: LEDs, Buttons & Switches 16. Serial Communication: UART →