Decision Making

Derivation 1: The DDM and Its Solution

The decision variable $x(t)$ accumulates evidence according to:

$$dx = \mu \, dt + \sigma \, dW$$

where $\mu$ is the drift rate (evidence strength), $\sigma$ is the diffusion coefficient (noise), and $dW$ is a Wiener process increment. The decision is made when $x(t)$ first reaches either boundary: $+a$ (choice A) or $-a$ (choice B), starting from $x(0) = z$. The probability of choosing A is:

$$P(A) = \frac{1 - \exp(-2\mu z / \sigma^2)}{1 - \exp(-2\mu a / \sigma^2)}$$

For symmetric boundaries ($z = 0$), this simplifies to:

$$P(A) = \frac{1}{1 + \exp(-2\mu a / \sigma^2)}$$

This is a logistic function of the signal-to-noise ratio $\mu a / \sigma^2$. The mean decision time for the correct response is:$\langle T \rangle = \frac{a}{\mu} \tanh\left(\frac{\mu a}{\sigma^2}\right)$, which decreases with evidence strength $\mu$ and increases with threshold $a$.

Derivation 2: Optimal Threshold and Reward Rate Maximization

If the decision-maker receives reward $R$ for correct responses and penalty$-C$ for errors, with a non-decision time $T_{\text{nd}}$ (sensory encoding + motor execution), the reward rate is:

$$\rho(a) = \frac{R \cdot P_c(a) - C \cdot (1 - P_c(a))}{\langle T(a) \rangle + T_{\text{nd}}}$$

Taking $d\rho/da = 0$ yields the optimal threshold. For the DDM with symmetric boundaries:

$$\frac{(R+C) \frac{dP_c}{da}}{\langle T \rangle + T_{\text{nd}}} = \rho^* \cdot \frac{d\langle T \rangle}{da}$$

This implicit equation shows that the optimal threshold increases with the reward-to-cost ratio $R/C$ and with non-decision time $T_{\text{nd}}$ (because time is already "wasted" on non-decision processes, it pays to be more accurate). Experimentally, subjects adjust their threshold in response to speed vs. accuracy instructions, with corresponding changes in LIP neural thresholds (Heitz and Schall, 2012).

Derivation 3: Mutual Inhibition Model of Evidence Accumulation

A biologically plausible implementation uses two competing neural populations with mutual inhibition. Let $r_1, r_2$ be the firing rates of populations favoring choices 1 and 2:

$$\tau \frac{dr_1}{dt} = -r_1 + f\left(w_{EE} r_1 - w_{EI} r_2 + I_1 + \sigma_n \eta_1(t)\right)$$

$$\tau \frac{dr_2}{dt} = -r_2 + f\left(w_{EE} r_2 - w_{EI} r_1 + I_2 + \sigma_n \eta_2(t)\right)$$

where $I_1 - I_2$ represents the evidence favoring choice 1, and $f$ is a nonlinear transfer function. Near the bifurcation point, the difference$\Delta r = r_1 - r_2$ follows approximately a DDM:

$$\tau_{\text{eff}} \frac{d\Delta r}{dt} \approx (I_1 - I_2) + (w_{EE} - w_{EI} - 1)\Delta r + \text{noise}$$

When $w_{EE} - w_{EI} < 1$ (stable regime), the system integrates evidence linearly. The effective time constant $\tau_{\text{eff}} = \tau / (1 - w_{EE} + w_{EI})$determines the integration timescale, which can be much longer than the membrane time constant, explaining slow evidence accumulation over hundreds of milliseconds.

Derivation 4: Softmax Action Selection from Value Estimates

Given learned values $Q(a_i)$ for actions $a_1, \ldots, a_K$, the brain must select an action. The softmax (Boltzmann) policy balances exploitation and exploration:

$$P(a_i) = \frac{\exp(\beta \, Q(a_i))}{\sum_{j=1}^{K} \exp(\beta \, Q(a_j))}$$

where $\beta$ is the inverse temperature. The values are updated via TD learning:

$$Q(a_i) \leftarrow Q(a_i) + \alpha \left[r - Q(a_i)\right]$$

The RPE $\delta = r - Q(a_i)$ matches the phasic dopamine response: firing above baseline for unexpected rewards ($\delta > 0$), below baseline for omitted rewards ($\delta < 0$), and at baseline for fully predicted rewards ($\delta = 0$). The explore-exploit balance controlled by $\beta$ maps to prefrontal cortex modulation of decision circuits.

Derivation 5: Urgency Signal and Collapsing Bounds

In many real decisions, deliberation time is costly. The urgency-gating model proposes that evidence is multiplied by a time-dependent urgency signal $u(t)$:

$$x(t) = u(t) \cdot \int_0^t e(s) \, ds$$

Equivalently, the decision can be modeled with collapsing (time-dependent) boundaries:

$$a(t) = a_0 \exp(-t / \tau_{\text{collapse}})$$

The optimal collapse rate depends on the prior distribution of difficulties. For a Bayesian decision-maker with a known prior over drift rates $p(\mu)$, the optimal boundary satisfies:

$$a^*(t) = \arg\max_a \left[\frac{\langle R \cdot P_c(\mu, a) \rangle_\mu}{\langle T(\mu, a) \rangle_\mu + T_{\text{nd}}}\right]$$

When easy and hard trials are intermixed, collapsing bounds outperform fixed bounds because they prevent excessive time on impossible trials. Neural evidence for urgency signals has been found in the supplementary eye field and caudate nucleus.