Machine Learning

Part VI: Sequence Models

Language, speech, time series, and video are all sequences: order matters and context accumulates. This part traces the evolution of sequence modelling from recurrent networks through the attention revolution to the large language models that now power modern AI. Every architecture is derived from first principles, with the mathematical intuition for why each innovation was necessary.

Chapter 16: Recurrent Networks

RNNs for sequential data, BPTT derivation of vanishing gradients, LSTM gates with full equations, GRU simplification, and bidirectional architectures.

RNN: h_t = tanh(W_hh h_{t-1} + W_xh x_t + b)BPTT: vanishing gradient as product of JacobiansLSTM: forget/input/output gates full equationsGRU & bidirectional RNNs

Chapter 17: Attention & Transformers

Scaled dot-product attention derived from first principles, multi-head attention, sinusoidal positional encoding, and the full Transformer encoder-decoder.

Attention(Q,K,V) = softmax(QKᵀ/√d_k)VWhy scale by √d_k: variance of dot productMulti-head attention & positional encodingTransformer encoder/decoder & layer norm

Chapter 18: Large Language Models

Autoregressive language modelling, GPT and BERT architectures, scaling laws, RLHF training pipeline, emergent abilities, and tokenisation.

Autoregressive: P(x_1,...,x_T) = Π P(x_t|x_{<t})GPT (decoder-only) vs BERT (bidirectional)Scaling laws: loss ~ N^(-α)RLHF: reward model + PPO fine-tuning

What you will learn

✓Derive backpropagation through time (BPTT) and explain the vanishing gradient problem

✓Write out all four LSTM gate equations and explain what each gate controls

✓Derive scaled dot-product attention and prove why scaling by √d_k stabilises training

✓Build a multi-head attention layer and understand its role in the Transformer

✓Construct sinusoidal positional encodings and explain their properties

✓Distinguish GPT (causal) from BERT (bidirectional) training objectives

✓Explain the empirical scaling laws for language models

✓Describe the RLHF pipeline: reward model, PPO, and alignment

Prerequisites

Part III: Deep Learning

Backpropagation, chain rule, feedforward networks, batch normalisation

Part I: Linear Algebra

Matrix multiplication, softmax, and the geometry of dot products

Part I: Probability

Cross-entropy, KL divergence, and language model perplexity

← Ch 15: Variational Inference Chapter 16: Recurrent Networks