10. Barrier Functions, Reachability & Safety Filters

Forward invariance, CBF-QPs, high-order and robust CBFs, HJ reachability, predictive safety filters, shields and the unified safety-filter view

Book contents · Apply this chapter to a real decision · Glossary

Modules 4–9 made safety a property of the learner: a safe set in parameter space, a constrained MDP, a trust region. This module makes safety a property of a runtime filter wrapped around any controller, including an RL policy (see Primer E) that knows nothing about safety. The filter watches the state, checks whether the proposed input keeps a certified set invariant, and minimally corrects it if not. Three families do this: control barrier functions (a one-step, gradient-based check), Hamilton–Jacobi reachability (the exact maximal safe set, computed offline) and predictive safety filters (an MPC-like search for a backup plan). The unifying view of Hsu, Hu & Fisac, Annu. Rev. 2024 shows they are all instances of one theorem: keep a fallback policy, monitor whether it can still rescue you, and intervene only when it could not.

1. Forward Invariance and Nagumo's Theorem

Safety in this module means one thing: the state never leaves a prescribed set. Everything else (CBFs, reachability, predictive filters, shields) is a way to certify or enforce that property, so we start with the property itself. Consider $\dot x=f(x)$ with $f$ locally Lipschitz (see Primer B), so that every $x_0$ has a unique solution on a maximal interval $I(x_0)=[0,\tau_{\max})$, with $\tau_{\max}=\infty$ if the system is forward complete (see Primer D).

Nagumo is a test, not a design tool, and it is fragile: it constrains $\dot h$ only on the boundary and says nothing about how fast trajectories may approach it, so a small model error or a sampled implementation can push the state over. Two ideas repair this. The first predates CBFs and is the verification ancestor of everything on this page.

The certificate's sign convention is the mirror image of ours ($B\approx -h$), and its third condition ($\dot h\ge 0$ everywhere) is much stronger than Nagumo's. The second idea, due to Ames, Xu, Grizzle and Tabuada, interpolates between the two: allow $h$ to decrease, but at most at a rate that vanishes as the boundary is approached.

2. Control Barrier Functions

Barrier functions verify a closed loop; control barrier functions let us design one, exactly as Sontag's control Lyapunov functions extend Lyapunov functions. Take the control-affine system $\dot x=f(x)+g(x)u$ with $f,g$ locally Lipschitz, $x\in D\subset\mathbb R^n$, $u\in U\subseteq\mathbb R^m$. Along any trajectory $$\dot h(x,u)=L_fh(x)+L_gh(x)\,u,$$ which is affine in $u$. This single fact is why CBF conditions become linear constraints and safety filters become quadratic programs.

Reciprocal versus zeroing barriers

The first CBF-QP paper, Ames, Grizzle & Tabuada, CDC 2014, used a barrier in the optimization sense: $B(x)=1/h(x)$ or $B=-\log\frac{h}{1+h}$, which blows up on $\partial C$, with the condition $\inf_u[L_fB+L_gBu-\gamma/B]\le 0$. The journal version Ames, Xu, Grizzle & Tabuada, TAC 2017 introduced both notions side by side.

Mediating performance and safety: the CLF-CBF QP

Given an exponentially stabilizing CLF $V$ (condition $\inf_u[L_fV+L_gVu+c_3V]\le 0$) and a barrier, the TAC 2017 paper unifies both in one quadratic program (see Primer B) (eqs. 33–34), here in the zeroing form of the 2019 tutorial:

The stability constraint is relaxed by a slack $\delta$ (see Primer B) penalized with $p>0$; the safety constraint is hard. When both cannot be met, performance yields and safety wins. Theorem 3 of the TAC paper (stated for the RCBF form $L_fB+L_gBu-\alpha(h)\le 0$) proves that if $f,g,\nabla B,\nabla V$ and the cost terms $H,F$ are locally Lipschitz (the TAC objective has an extra linear term $F(x)^\top u$) and $L_gB(x)\neq 0$ on $\operatorname{Int}(C)$, then $u^*(x)$ is locally Lipschitz, so the invariance theorem applies to the closed loop and an explicit closed form exists.

3. The CBF-QP Safety Filter and Its Closed Form

Suppose a nominal controller $k(x)$ (a tracking law, a planner, an RL policy) is given and may violate the CBF condition. For unrestricted inputs ($U=\mathbb R^m$) and $L_gh(x)\neq 0$, $K_{\mathrm{cbf}}(x)$ is a half-space in $u$, so the minimally invasive repair is a projection:

Four things the closed form tells you

Lipschitz continuity. $r\mapsto\max\{0,r\}$ is 1-Lipschitz, and products and quotients of locally Lipschitz functions are locally Lipschitz wherever the denominator is bounded away from zero. Hence $u^*$ is locally Lipschitz wherever $a,b,k$ are and $b(x)\neq 0$, which is exactly the hypothesis $L_gB\neq 0$ of Theorem 3 in Ames et al. 2017. This is not a technicality: the invariance theorem needs a Lipschitz closed loop, so an implementation that switches discontinuously between controllers forfeits the guarantee unless it is analysed separately.

Feasibility with input bounds. With $U=\mathbb R^m$ and $b(x)\neq 0$ the half-space is always non-empty. With bounds, the QP is feasible iff $K_{\mathrm{cbf}}(x)\cap U\neq\emptyset$, which agrees with the CBF supremum test when the finite supremum is attained (in particular for nonempty compact $U$). The bounds must then be inside the QP: clipping the unconstrained solution afterwards can violate the constraint even when $K_{\mathrm{cbf}}(x)\cap U\neq\emptyset$ (the explorer below has an example). A function that is a CBF for $U=\mathbb R^m$ need not be one for a box $U$: a double integrator at distance $\Delta$ from a wall with approach speed $v$ needs $|u|\ge v^2/(2\Delta)$ to stop, whatever $\alpha$ says. Ames et al. 2017 (Rem. 11) note that their constructions for higher relative degree may cease to be valid once $U\neq\mathbb R^m$ and list CBF design under input constraints as an open question; the remedies are the high-order constructions of Section 4, predictive filters (Section 7), and value-function CBFs that respect the bounds by construction (Section 9). The other extreme is instructive too: for a drift-free single integrator $\dot x=u$ the input $u=0$ always satisfies $L_gh\,u\ge-\alpha(h)$ when $h\ge 0$, and scaling any feasible $u$ towards zero keeps it feasible, so a norm bound $\|u\|\le u_{\max}$ can never make the QP infeasible. Input constraints only bite when the drift $L_fh$ works against you, which is why the explorer includes a drift term.

In general $K_{\mathrm{cbf}}(x)=U\cap\{u:\ a(x)+b(x)^\top u\ge 0\}$, a half-space only when $U=\mathbb R^m$. Attainment is not a technicality: for $U=(-1,0)$ and the constraint $u\ge 0$ at a boundary state ($a=0$, $b=1$) the supremum of $a+bu$ over $U$ is $0\ge-\alpha(0)$, so the supremum test passes, yet no admissible input satisfies the constraint. What the filter needs is an actual element of $K_{\mathrm{cbf}}(x)$.

Sampled data. The theory is continuous-time; the QP is solved every $\Delta t$ and the input is held. Between samples $h$ can undershoot. Hsu, Hu & Fisac (Cor. 1.2) argue safety under $\alpha(r)\lt r/\Delta t$ (for $\alpha(r)=\alpha_0 r$: $\alpha_0\Delta t\lt 1$) through the first-order step $h(x')\ge h(x)-\alpha(h(x))\Delta t$, which neglects the integration error between control cycles (their footnote 8, which adds that practical implementations typically enforce a positive margin such as $\dot h\ge\epsilon>0$). The paper prints the condition "for all $a\in\mathbb R$", which cannot hold at $a=0$ and reverses for $a\lt 0$; only arguments $a=h(x)\ge 0$ enter the proof, where it gives $h(x')\gt 0$ if $h(x)\gt 0$ and $h(x')\ge 0$ at $h(x)=0$. It is therefore a first-order argument, not a sampled-data guarantee: an exact one needs a bound on the inter-sample error or a genuinely discrete-time barrier condition. The cleaner route is a discrete-time CBF: Cheng et al., AAAI 2019 (Def. 1) require $\sup_{a}\big[h(f(s)+g(s)a+d(s))+(\eta-1)h(s)\big]\ge 0$ with $\eta\in[0,1]$, i.e. $h(s_{t+1})\ge(1-\eta)h(s_t)$, whose discrete comparison argument is one line: $h_t\ge(1-\eta)^th_0\ge 0$.

Slack. Adding a slack $\epsilon\ge 0$ with a large penalty (as in Cheng et al.'s QP (9)) guarantees a solution always exists, but what is then invariant, for $0\lt\eta\le 1$ and a slack bound $\epsilon\le\epsilon_{\max}$ on that set, is the enlarged set $\{h\ge-\epsilon_{\max}/\eta\}$ (their Lemma 1); for $\eta=0$ the slack can accumulate, $h_{t+1}\ge h_t-\epsilon_{\max}$, and no fixed inflation holds. Graceful degradation, not safety.

4. High Relative Degree: ECBFs and HOCBFs

Iterated Lie derivatives mean $L_f^0h=h$ and $L_f^{j+1}h=\nabla(L_f^jh)^\top f$; $L_gL_f^jh=\nabla(L_f^jh)^\top g$. Here $f(x)\in\mathbb R^n$ and $g(x)\in\mathbb R^{n\times m}$, so the latter is a row with $m$ entries. The notation $C^r$ means that derivatives through order $r$ exist and are continuous; assume the dynamics and barrier are smooth enough for all derivatives written below.

The relative degree of $h$ with respect to $\dot x=f+gu$ is the number $r$ of times one must differentiate $h$ along the dynamics before the input appears: $L_gL_f^kh\equiv 0$ for $k=0,\dots,r-2$ and $L_gL_f^{r-1}h\neq 0$, so that $$h^{(r)}(x,u)=L_f^rh(x)+L_gL_f^{r-1}h(x)\,u.$$ Most robotic constraints have $r\ge 2$: a position constraint under a force input, a distance constraint for a vehicle whose input is acceleration. Then $L_gh\equiv 0$ and the CBF constraint $L_fh+\alpha(h)\ge 0$ contains no $u$ at all: $K_{\mathrm{cbf}}(x)$ is either all of $U$ or empty (Ames et al. 2017, Sec. III-C), and the closed form divides by $b^\top b=0$. The fix is to push the constraint down the chain of derivatives.

Why it works. Treat $\mu=h^{(r)}$ as a virtual input to a chain of $r$ integrators. With $\mu=-K_\alpha\eta_b$ the barrier obeys a linear ODE, $h(t)=C_ce^{(F-GK_\alpha)t}\eta_b(x_0)$; with $\mu\ge-K_\alpha\eta_b$ the excess $\mu+K_\alpha\eta_b\ge 0$ enters $h$ through the impulse response of the closed-loop chain, which is a convolution of decaying exponentials and hence nonnegative when the poles are real, so the inequality follows (with complex poles the impulse response changes sign and it can fail). The gains are chosen by pole placement (see Primer D), but not arbitrarily: define $\nu_0=h$, $\nu_i=\dot\nu_{i-1}+p_i\nu_{i-1}$ and $C_i=\{\nu_i\ge 0\}$, where $-p_1,\dots,-p_r$ are the eigenvalues of $F-GK_\alpha$. Then (Ames et al. 2019, Prop. 6 and Thms. 7–8, from Nguyen & Sreenath): if $C_i$ is forward invariant, $p_i>0$ and $x_0\in C_i\cap C_{i-1}$, then $C_{i-1}$ is forward invariant; hence if the controller enforces $\nu_r(x,u)\ge0$ at every time and $\nu_0(x_0),\ldots,\nu_{r-1}(x_0)\ge0$, then $C$ is invariant. The initial-condition requirement $\nu_i(x_0)=\dot\nu_{i-1}(x_0)+p_i\nu_{i-1}(x_0)\ge 0$, i.e. $p_i\ge-\dot\nu_{i-1}(x_0)/\nu_{i-1}(x_0)$ when $\nu_{i-1}(x_0)\gt 0$, says the pole magnitudes must be large enough relative to how fast you are initially moving towards the boundary (if $\nu_{i-1}(x_0)=0$ it reads $\dot\nu_{i-1}(x_0)\ge 0$, which no choice of $p_i$ can repair). Only $\nu_0,\dots,\nu_{r-1}$ are initial-state tests; $\nu_r$ contains $u$ and is the input constraint the controller enforces. (The ECC tutorial is not consistent here: it first calls $p_1,\dots,p_r$ the roots of the characteristic polynomial, then requires $p_i>0$ for "real negative poles", and Theorem 8 prints the initial-condition inequality with the eigenvalue $\lambda_i(F-GK_\alpha)\ge-\dot\nu_{i-1}(x_0)/\nu_{i-1}(x_0)$. Read consistently, with $-p_i$ the poles, the condition is $p_i\ge-\dot\nu_{i-1}(x_0)/\nu_{i-1}(x_0)$, as Exercise 10.3 checks on a concrete case.)

5. Robust CBFs, ISSf and Learned Residuals

Everything so far assumed the model is exact. With a model error the true barrier rate is $\dot h=L_fh+L_ghu+\nabla h^\top d$ for an unmatched error $\dot x=f+gu+d$, or $\dot h=L_fh+L_gh(u+d)$ for a matched error entering through the input channel. A CBF condition certified on the model can then fail by $|\nabla h^\top d|$. There are two responses: robustify the condition (accept a slightly larger invariant set), or learn the error and shrink it.

Input-to-state safety

Kolathaya & Ames, L-CSS 2019 consider a safeguarding controller $k(x)$ applied with a bounded additive input disturbance, $\dot x=\bar f(x)+g(x)d(t)$ with $\bar f:=f+gk$ and $\|d\|_\infty\le\bar d$, and ask for the safety analogue of input-to-state stability: invariance of a set that inflates with the disturbance and approaches $C$ as the chosen disturbance bound tends to zero.

Learning the residual of $\dot h$

The alternative is to learn the error where it matters, which is only inside $\dot h$. Taylor, Singletary, Yue & Ames, L4DC 2020 note that with a nominal model $\hat f,\hat g$ the true barrier rate is $$\dot h(x,u)=\underbrace{\nabla h^\top(\hat f+\hat g u)}_{\text{nominal}}+\underbrace{\nabla h^\top\big((f-\hat f)+(g-\hat g)u\big)}_{=:\ b(x)+a(x)^\top u},$$ still affine in $u$ (here $a,b$ are Taylor et al.'s residual terms, not the $a,b$ of Section 3). They fit $\hat a,\hat b$ by empirical risk minimization (see Primer E) on data $(x_i,u_i,\dot h_i)$ (with $\dot h_i$ estimated from the measured trajectories), use $\widehat{\dot h}$ inside the LCBF-QP $\operatorname{arg\,min}\tfrac12\|u-k_d(x)\|^2$ s.t. $\widehat{\dot h}(x,u)\ge-\alpha(h)$, then run the filter, aggregate the new data DAgger-style and refit. The constraint stays affine and the QP stays convex; with a bound on the residual error, the ISSf argument above gives invariance of an inflated set. Validated on a Segway.

Cheng, Orosz, Murray & Burdick, AAAI 2019 (RL-CBF) do the same in discrete time with a GP: $s_{t+1}=f(s_t)+g(s_t)a_t+d(s_t)$, $|\mu_d(s)-d(s)|\le k_\delta\sigma_d(s)$ with probability $1-\delta$ (GP band, Module 3), affine $h=p^\top s+q$, and the compensator problem (their eq. 14, which the paper calls a QP although it prints the unsquared norm) $$\min_{a,\epsilon}\ \|a\|_2+K_\epsilon\,\epsilon\quad\text{s.t.}\quad p^\top f+p^\top g\,(u^{RL}_\theta(s_t)+a)+p^\top\mu_d(s_t)-k_\delta|p|^\top\sigma_d(s_t)+q\ \ge\ (1-\eta)h(s_t)-\epsilon,\quad a_{\mathrm{low}}\le a+u^{RL}_\theta\le a_{\mathrm{high}},\quad \epsilon\ge 0,$$ applied as $u=u^{RL}_\theta+u^{CBF}$. The paper leaves $\epsilon\ge 0$ implicit; without it the linear penalty rewards negative slack, i.e. an over-tightened constraint and a needlessly large correction. A large $K_\epsilon$ ($10^{12}$ in the paper) makes slack expensive but does not by itself force $\epsilon=0$ whenever the hard constraint can be met: this $\ell_1$ penalty is exact only if $K_\epsilon$ exceeds the Lagrange multiplier of the hard-constrained problem, which is $1/\|g^\top p\|$ for the unsquared norm without active bounds and is unbounded as the input loses authority over $h$ (for instance $\min_{0\le a\le 1}|a|+10^{12}\epsilon$ s.t. $10^{-13}a\ge 10^{-13}-\epsilon$, $\epsilon\ge 0$ is solved by $a=0$, $\epsilon=10^{-13}$ at cost $0.1$, not by the feasible $a=1$, $\epsilon=0$ at cost 1). Zero slack whenever possible needs a verified penalty bound, or solving the hard-constrained QP first and relaxing only if it is infeasible. With $\epsilon$ fixed to 0, $\|a\|_2$ and $\|a\|_2^2$ have the same minimizer; with slack their exactness thresholds differ ($1/\|g^\top p\|$ versus $2\|a^\star\|/\|g^\top p\|$), so the two objectives can differ even when the hard constraint is feasible. Lemma 1: if a solution with $\epsilon=0$ exists on all of $C$, $C$ is forward invariant with probability $1-\delta$; for $0\lt\eta\le 1$, if $\epsilon\le\epsilon_{\max}$ at every state of the enlarged set $\{h\ge-\epsilon_{\max}/\eta\}$, that set is forward invariant (the proof applies Definition 1 to $h+\epsilon_{\max}/\eta$). For $\eta=0$ no finite inflation follows: $h_{t+1}\ge h_t-\epsilon_{\max}$ lets $h$ drift down by $\epsilon_{\max}$ per step. The second contribution is to feed past CBF corrections back into the policy so the RL agent learns to stop proposing unsafe actions (TRPO or DDPG; inverted pendulum and car-following).

Learning the barrier itself

Robey et al., CDC 2020 learn $h_\theta$ from safe expert trajectories by a margin program (trained like a neural network, see Primer E), $$\min_\theta\ \|\theta\|^2+\lambda_s\!\!\sum_{x_i\in\bar X_{\mathrm{safe}}}\!\![\gamma_{\mathrm{safe}}-h_\theta(x_i)]_++\lambda_u\!\!\sum_{x_i\in X_N}\!\![h_\theta(x_i)+\gamma_{\mathrm{unsafe}}]_++\lambda_d\!\!\sum_{(x_i,u_i)\in Z_{\mathrm{dyn}}}\!\!\big[\gamma_{\mathrm{dyn}}-\langle\nabla h_\theta(x_i),f(x_i,u_i)\rangle-\alpha(h_\theta(x_i))\big]_+,$$ which is convex when $h_\theta$ is linear in $\theta$ and $\alpha$ is linear (then every hinge has an affine argument; their Sec. 3.4.1 makes this point for the constrained version of the program, whose Lipschitz constraints are checked afterwards). For DNNs or a nonlinear $\alpha$ it is non-convex and is trained by stochastic gradient methods. They prove with Lipschitz arguments on $\epsilon$-nets that if the samples are dense enough relative to the margins (schematically $\epsilon\lesssim\gamma/L$), the learned function is a valid CBF on the sampled region. The hinge form is reused by much of the later neural-CBF training, including the multi-agent GCBF+ of Section 9 and many of the certificates surveyed in Module 11; the PNCBF of Section 9 is a notable exception, since it regresses a policy value instead.

Finally, Choi, Lee, Sreenath, Tomlin & Herbert, CDC 2021 build the bridge to Section 6. Their robust control barrier-value function $$B_\gamma(x,t):=\min_{\xi_d\in\Xi_{[t,0]}}\ \max_{u(\cdot)\in\mathcal U_{[t,0]}}\ \min_{s\in[t,0]}e^{\gamma(s-t)}\,l(x(s))$$ is the HJ safety value of Section 6 with an exponential weight. It satisfies the CBVF variational inequality (see Primer D) $$0=\min\Big\{l(x)-B_\gamma(x,t),\ D_tB_\gamma+\max_{u\in U}\min_{d\in\mathcal D}D_xB_\gamma\cdot f(x,u,d)+\gamma B_\gamma(x,t)\Big\},\qquad B_\gamma(x,0)=l(x),$$ in the viscosity sense made precise below.

The safe-input set is $K_{B_\gamma}(x,t)=\{u\in U: D_tB_\gamma+\min_{d}D_xB_\gamma\cdot f(x,u,d)+\gamma B_\gamma\ge 0\}$ (their eq. 16). For dynamics affine in control and disturbance, $\dot x=p(x)+q(x)u+r(x)d$ (their eq. 18), the minimum over $d$ does not depend on $u$, so this set is $U$ intersected with a half-space in $u$, and with a polytopic $U$ the robust CBVF-QP is feasible wherever $D_xB_\gamma$ exists (Prop. 3). Unlike the least-restrictive switch, which acts only at the boundary, this filter adjusts any reference input everywhere inside $S(t)$ (their Remark 3); the paper claims nothing about the regularity of the resulting feedback, and where the gradient does not exist it falls back on super- or subdifferentials (Remark 4). In this precise sense $B_\gamma$ is a CBF with $\alpha(r)=\gamma r$ whose safe set is the largest possible one, and $\gamma=0$ recovers the HJ value itself.

6. Hamilton–Jacobi Reachability

A CBF certifies a set you propose. Reachability computes the largest set that can be kept safe, together with the controller that does it, by solving a differential game offline. Consider $\dot x=f(x,u,d)$ with control $u\in U$ and disturbance $d\in\mathcal D$ (model error, an adversary, wind). The disturbance plays with non-anticipative strategies $\beta\in\mathcal B$: $\beta[u](t)$ may depend on $u$ only up to time $t$, which gives it the information advantage that makes the analysis worst-case.

Targets and tubes. The founding formulation of Mitchell, Bayen & Tomlin, TAC 2005 is phrased for a target $\{g\le 0\}$ (the unsafe set) and computes the set of states from which the minimizing player can force entry into it within time $\tau$, against the maximizing player: the zero sublevel set $\mathcal G(\tau)=\{x: v(x,-\tau)\le 0\}$ of the viscosity solution of $$D_tv(x,t)+\min\big[0,\ H(x,D_xv(x,t))\big]=0,\qquad v(x,0)=g(x),\qquad H(x,p)=\max_{a\in A}\min_{b\in B}p^\top f(x,a,b).$$ The $\min[0,\cdot]$ freezes $v$ once a trajectory has entered the target, which turns a reachable set (target hit at exactly time $\tau$) into a reachable tube (hit at some time within $\tau$). The tutorial of Bansal, Chen, Herbert & Tomlin, CDC 2017 writes the BRS as $\mathcal G(t)=\{x: G(t,x)\le 0\}$ with $D_tG+H(t,x,\lambda)=0$, $G(0,x)=g(x)$, $H=\max_{a}\min_{b}\lambda\cdot f(x,a,b)$ (eqs. 11–13), the optimal control $a^*=\operatorname{arg\,max}_a\min_b\lambda\cdot f$ (eq. 14), and the BRT as $$\mathcal G(t)=\{x:\ \exists\beta\in\Gamma(t),\ \forall a(\cdot)\in\mathcal A,\ \exists s\in[t,0]:\ \zeta(s;x,t,a(\cdot),\beta[a](\cdot))\in\mathcal G_0\}\qquad\text{(eq. 16)},$$ computed from a final-value PDE similar to the BRS one, for instance Mitchell's $\min\{0,H\}$ form. Their rule of thumb: whenever a set asks "there exists a control", that input is minimized in the Hamiltonian; "for all controls" means maximized. The safe set is the complement of the BRT of the failure set, and Fisac's $\{V\ge 0\}$ is the same object with $l=g$ (both margins are positive on the safe side), the horizon written as $[0,T]$ with $V(x,T)=l(x)$ instead of $[-\tau,0]$ with $v(x,0)=g(x)$, and one boundary convention swapped: Mitchell's target $\{g\le 0\}$ is closed, so the complement of the BRT is the open set $\{v\gt 0\}$, whereas Fisac's safe set $\{V\ge 0\}$ is closed and counts the level set $\{V=0\}$ as safe.

The least-restrictive filter and the Fisac et al. safety framework

Because every superlevel set of $V$ is invariant, the filter can be lazy: apply any learning controller $\kappa_l$ while $V(x)>0$ and switch to $\kappa^*$ only on the boundary. Fisac et al., TAC 2019 wrap this around an arbitrary RL controller with two additions. First, the disturbance bound is learned: a GP on the unmodelled dynamics $d(x)$ gives a state-dependent set $\hat{\mathcal D}(x)$ containing $d(x)$ with probability $p$, and reachability is run with $d\in\hat{\mathcal D}(x)$. Second, the guarantee is monitored: their Prop. 5 shows that a level set which is robustly invariant under $\hat{\mathcal D}$ stays invariant if the true $d$ lies in $\hat{\mathcal D}$ merely on that level set's boundary, so what must be believed is not "the model is right everywhere" but "the model is right on some level set we can retreat to". The posterior probability of that event, the global safety confidence $\gamma(x)$ (their eq. 14: the posterior probability that some level set $\{V=\alpha\}$ with $0\le\alpha\le V(x)$ has $d\in\hat{\mathcal D}$ everywhere on it), drives the least-restrictive law (eq. 15) $$\kappa(x)=\begin{cases}\kappa_l(x), & \gamma(x)>\gamma_0\ \wedge\ V(x)>0,\\ \kappa^*(x), & \text{otherwise,}\end{cases}$$ with a cheaper local variant based on the reliability of $\hat{\mathcal D}$ at the current state (their Prop. 1, which bounds how long a locally reliable model stays reliable by Lipschitz constants). In experiments a quadrotor learning by policy gradient never crashes, including under an unmodelled fan. The switching itself is discontinuous and can chatter; the CBVF-QP of Section 5 is the minimally invasive alternative that acts everywhere inside the safe set rather than only on its boundary.

Bridging to reinforcement learning

Grid-based HJ solvers scale exponentially in $n$ (Bansal & Tomlin put their direct use at up to about five state dimensions). Fisac, Lugovoy, Rubies-Royo, Ghosh & Tomlin, ICRA 2019 observed that the undiscounted safety Bellman equation $V(x)=\min\{l(x),\max_uV(f(x,u))\}$ is not a contraction (see Primer E), so temporal-difference methods have no convergence guarantee, but a discounted version is:

Reach-avoid. Hsu et al. (RSS 2021) add liveness: with $l(s)\le 0\iff s\in\mathcal T$ (target) and $g(s)>0\iff s\in\mathcal F$ (failure), the reach-avoid set $RA=\{V\le 0\}$ is characterized by the reach-avoid Bellman equation (their eq. 10) $$V(s)=\max\Big\{g(s),\ \min\big\{l(s),\ \min_{u}V(s^u_+)\big\}\Big\},$$ and its discounted version (eq. 15) $$V_\gamma(s)=\gamma\max\Big\{\min\big\{\min_uV_\gamma(s^u_+),\,l(s)\big\},\,g(s)\Big\}+(1-\gamma)\max\{l(s),g(s)\}$$ is a sup-norm contraction for every $\gamma\in[0,1)$ (Prop. 1), tabular Q-learning on a finite discretization converges to it with probability 1 (Prop. 2, via the standard Q-learning conditions), its fixed point converges to that of the undiscounted equation as $\gamma\to 1$ (Prop. 3), and (Thm. 1) $RA_{\gamma_1}\subseteq RA_{\gamma_2}\subseteq RA$ for $0\le\gamma_1\le\gamma_2\lt 1$, with $RA_\gamma\to RA$ in Hausdorff distance as $\gamma\to 1$ provided $RA$ has non-empty interior and $V$ is nowhere locally constant on its zero level set. The deep-RL solution is then used as an untrusted oracle inside a supervisory filter that keeps the zero-violation guarantee.

Neural PDE solvers. DeepReach (Bansal & Tomlin, ICRA 2021) trains a sine-activation network $V_\theta(x,t)$ self-supervised on the variational inequality itself, $$\mathcal L=\big\|V_\theta(x_i,t_i)-l(x_i)\big\|\,\mathbb 1(t_i=T)+\lambda\,\Big\|\min\{D_tV_\theta+H(x_i,t_i),\ l(x_i)-V_\theta(x_i,t_i)\}\Big\|,\qquad H=\max_u\min_d\langle\nabla V_\theta,f\rangle,$$ where the indicator $\mathbb 1(t_i=T)$ is 1 for terminal-time samples and 0 otherwise (see Primer C), with a weight $\lambda\gt 0$ and a curriculum that grows the horizon backward from $t=T$, so the cost scales with the complexity of $V$ rather than with a grid (9-D multi-vehicle and 10-D narrow-passage examples). It is a physics-informed approximation with no formal guarantee. The connection to Module 7 is exact: $\{V\ge 0\}$ is the viability (discriminating) kernel, and the safe value functions of Massiani et al. are the penalty route to the same set.

7. Predictive Safety Filters (Wabersich & Zeilinger)

A CBF-QP looks one derivative ahead and needs an explicit invariant set $C$. A predictive safety filter looks $N$ steps ahead (see Primer D) with a model and needs only a small terminal safe set: the invariant set it enforces is implicit, namely the set of states from which a feasible backup plan exists. Wabersich & Zeilinger, Automatica 2021 formulate it for discrete-time $x(k+1)=f(x(k),u(k);\theta_R)$ with constraints $X$, $U$, an RL input $u_L(k)=\pi_L(k,x(k))$ and the filtered input $u(k)=\pi_S(k,x(k),u_L(k))$. The filter turns the constrained system into the unconstrained "safe system" $f_S(k,x,u_L):=f(x,\pi_S(k,x,u_L))$ on which any RL algorithm may run. An input $u_L(\bar k)$ is certified (Def. 3.1) if $\pi_S(\bar k,x(\bar k),u_L(\bar k))=u_L(\bar k)$ and continuing with the filter keeps the system safe forever.

Uncertain models. Beyond $Z_c$ the paper replaces $f(\cdot;\bar\theta)$ by the mean model of a Bayesian parametric belief $p(\theta|\mathcal D)$ (see Primer C), predicts nominal states $\mu_{i|k}$, and tightens the constraints along the horizon: $\mu_{i|k}\in\bar X_i$, $v_{i|k}\in\bar U_i$, $\mu_{N|k}\in\bar S^t_N$, plus a model-confidence constraint $\mathcal E_{p_S}(\mu_{i|k},v_{i|k})\subseteq\bar{\mathcal E}^\gamma_i$ (problem (6)). Here $\mathcal E_{p_S}$ (Def. 4.4) is a set-valued map (see Primer 0) with $\Pr(\forall k: e(k)\in\mathcal E_{p_S}(x(k),u(k)))\ge p_S$ for the one-step prediction error $e$, obtained from a confidence region of the parameter posterior; $\gamma$ scales the admissible error set. Under a tracking assumption (Assumption 4.3: a policy $\pi$ and an incremental Lyapunov function $V_{\mathrm{inc}}$ (their $V$) with $c_l\|x-\mu\|^2\le V_{\mathrm{inc}}(x,\mu,v)\le c_u\|x-\mu\|^2$ that contracts at rate $\rho\in(0,1)$, $V_{\mathrm{inc}}^+\le\rho V_{\mathrm{inc}}$, in a neighbourhood of the nominal plan; under this nominal tracking assumption with no fresh disturbance, the deviation therefore decays like $\|x_i-\mu_i\|\le\sqrt{c_u/c_l}\,\rho^{i/2}\|x_0-\mu_0\|$, which is why the increments of the constraint tightening (their eq. 10) are $\epsilon\sqrt{\rho}^{\,i}$ for the tightening factor $\epsilon$ below) and Lipschitz continuity of $\mathcal E_{p_S}$ (Assumption 4.5), together with the terminal safe set of Assumption 4.2, Theorem 4.6 states: for a tightening factor $\epsilon>0$ and a sufficiently small Lipschitz constant $L_{\mathcal E_{p_S}}$ one can pick $\gamma>0$ such that initial feasibility of (6) implies $$\Pr\big(\forall k\in\mathbb I_{[0,\bar N]}:\ x(k)\in X,\ u(k)\in U\big)\ \ge\ p_S .$$ The mechanism is the one of tube MPC (Mayne, Seron & Raković, Automatica 2005; see Primer D): the tightening pays for the accumulated prediction error, and the confidence constraint forces backup plans to enter poorly modelled regions only "carefully". The whole guarantee is conditional on the Bayesian error model being calibrated; the GP versions of this statement (Koller et al. 2018) inherit the RKHS-norm caveats of Module 3.

Seen through this table, the CBF-QP is a predictive filter with $N=1$ whose terminal set is $C$ itself and whose one-step invariance test has been relaxed to $\dot h\ge-\alpha(h)$. The price of the horizon is the online NLP and the model; the reward is that the safe set is defined implicitly by feasibility, so input constraints, state constraints and model confidence all enter uniformly. Wabersich & Zeilinger, TAC 2023 close the remaining gap with predictive CBFs. An auxiliary soft-constrained problem with slacks $\xi_i\ge 0$ on tightened state constraints and a terminal CBF $h_f$, $h_{PB}(x)=\min_{u,\xi}\ \alpha_f\xi_N+\sum_{i\lt N}\|\xi_i\|$ subject to the dynamics and the slackened constraints (their (9), written out below), is feasible for every state. For $\alpha_f$ large enough, under the assumptions of the theorem below (their Thm. III.6), its optimal value is a discrete-time CBF in their sign convention ($h\le 0$ safe) on the domain $D_{PB}=\{x: h_{PB}(x)\le\alpha_f\gamma_f\}$, where the level $\gamma_f$ comes from the terminal CBF's domain: $\inf_{u\in U}h_{PB}(f(x,u))-h_{PB}(x)\le-\Delta h_{PB}(x)$ with a continuous $\Delta h_{PB}>0$ outside $S_{PB}=\{h_{PB}=0\}$. The recovery filter therefore renders $S_{PB}$ asymptotically stable with region of attraction $D_{PB}$; because of the tightening, $S_{PB}$ is only a subset of the original filter's feasible set. After a model error or disturbance that leaves the state inside $D_{PB}$, the system returns to safety instead of stalling. The learning-based MPC lineage behind all this, from the LBMPC of Aswani et al., Automatica 2013 (robust nominal tube for safety, learned oracle only in the cost) through the GP-ellipsoid safe MPC of Koller et al., CDC 2018 to the review of Hewing et al., Annu. Rev. 2020, is the subject of Module 11; the backup-policy idea of GoSafe (Module 6) is the same principle in parameter space.

8. Shields and the Unified Safety-Filter View

The formal-methods community arrived at the same architecture from discrete specifications. Alshiekh et al., AAAI 2018 define safe RL as learning an optimal policy while a temporal-logic safety specification $\varphi_s$ holds during learning and execution (their Def. 1), and enforce it with a shield, a reactive system synthesized by solving a safety game.

Read against Sections 3–7, the winning region $W$ is the maximal safe set of the abstraction (a discrete discriminating kernel), the preemptive shield is the exact analogue of $K_{\mathrm{cbf}}(x)$, and the post-posed shield is a least-restrictive filter whose "projection" is a ranking rather than a Euclidean distance. The abstraction plays the role of the model, and the guarantee is exactly as good as the abstraction. The survey and benchmark of Krasowski, Thumm, Müller, Schäfer, Wang & Althoff, TMLR 2023 sorts provably safe RL by how the action is adapted: action replacement (post-posed shields, switching filters), action projection (CBF-QPs, predictive filters) and action masking (preemptive shields). On their inverted-pendulum and quadrotor tasks replacement performed best, and a reward penalty whenever the safety verification intervenes improved training.

The universal safety filter theorem

Hsu, Hu & Fisac, Annu. Rev. 2024 distil every filter into three components acting on an information state $\eta\in\mathcal H$ (the state $x$ when it is fully observed): a fallback policy $\pi^{\mathrm{fb}}$, a safety monitor $\Delta^{\mathrm{fb}}$ and an intervention scheme $\phi$. Their Prop. 1 first records the ideal: with the maximal safe set $\Omega^*$ there exists a perfect (least-restrictive) filter that alters an action only if it would leave $\Omega^*$ in one step, and the safe decision problem becomes an unconstrained one on the filtered dynamics $u_k=\phi(x_k,\pi^{\mathrm{task}}(x_k))$. Everything else approximates it.

Two entries deviate from the paper's wording. The HJ row is the discrete-time filter of Hsu et al.'s eqs. 9–11, so its fallback maximizes the next value, $\pi^*=\operatorname{arg\,max}_u\min_dV(f(x,u,d))$ (their eq. 10). The gradient law $\kappa^*$ of Section 6 belongs to the continuous-time formulation and can leave the safe set in one discrete step: for $x^+=u$, $U=[-2,2]$ and $V(x)=1-x^2$ (the exact safety value for $l=V$), at $x=0.5$ it picks $u=-2$ and $V(x^+)=-3$, whereas $\pi^*$ picks $u=0$. The paper's monitor is $\min_dV(f(x,u,d))$ alone; the extra $l(x)$ makes Def. 1's "no current failure" clause hold at every state, not only along trajectories that start in $\Omega^*$. In the CBF-QP row, Hsu et al.'s implicit fallback $\operatorname{arg\,max}_u\dot h(x,u)$ exists only for compact $U$: with the module's $U=\mathbb R^m$ and $L_gh\neq 0$ the affine objective is unbounded. The proof of Cor. 1.2 needs only some admissible input satisfying the CBF inequality, e.g. the min-norm element of $K_{\mathrm{cbf}}(x)$ (the CBF-QP solution for $k\equiv 0$).

The survey of Wabersich, Taylor, Choi, Sreenath, Tomlin, Ames & Zeilinger, IEEE CSM 2023 takes the complementary, control-theoretic view: it defines the ideal filter, an optimal-control problem over whole input signals $v(\cdot)$ (see Primer D), $$u(\cdot)=\operatorname*{arg\,min}_{v(\cdot)}\int_0^\infty\|v(t)-u_{\mathrm{des}}(t)\|\,dt\quad\text{s.t.}\quad\dot x=f(x,v),\ x(0)=x_0,\ v(t)\in U,\ x(t)\in X\ \forall t\ge 0,$$ which is intractable because $u_{\mathrm{des}}(\cdot)$ is only revealed online and the problem is infinite-dimensional, and then develops HJ, CBF and predictive filters side by side as approximations of it, each with its data-driven variant (learned disturbance bounds for HJ as in Fisac et al., a learned $\dot h$ residual as in Taylor et al., a Bayesian model inside the PSF as in Section 7), a common design example, and hardware case studies. It is the single best reference for this module.

9. Learned and Uncertainty-Aware Filters (incl. UPSi)

The frontier is systems where neither $h$, nor $V$, nor even a state-space model is available in closed form. Four ideas from 2024–2026 cover it: learn a CBF by evaluating a policy, learn a game value by self-play, run reachability in a world model's latent space, and run a predictive filter on an ensemble of neural dynamics models.

So et al., ICRA 2024 (PNCBF) thereby sidestep the two failure modes of neural-CBF training, the unknown safe set and input constraints: the value is learned by regression on rollouts (loss (15): $\|V_\theta(x_t)-\max\{\max_{t\le s\le T}h(x_s),V_\theta(x_T)\}\|^2$), and because $\pi$ respects the actuator limits the resulting CBF-QP is feasible where the policy was. In practice the discounted variant $V_\lambda^{h,\pi}$ (their eq. 16) is used to make the fixed point unique, mirroring Section 6. Demonstrated up to a 16-D F-16 model and on two quadcopters. With $\pi=\pi^*$ the policy value is the HJ value, so PNCBFs sit between hand-designed CBFs and full reachability. GCBF+ (Zhang, So, Garg & Fan, T-RO 2025) extends learned CBFs to multi-agent systems: one graph-neural certificate $h_\theta$ on local neighbourhood graphs, trained with hinge losses on the CBF conditions in the spirit of Robey et al., is shared by all agents and scales to arbitrarily many of them (up to 20% better than the best hand-crafted CBF method with up to 256 agents and up to 40% better than leading RL methods at 1024, plus hardware experiments on a drone swarm).

Adversarial imagination. Gameplay filters (Nguyen, Hsu, Yu, Tan & Fisac, CoRL 2024) learn a zero-sum reach–avoid game by adversarial RL in simulation: a safety policy $\pi^{\mathrm{safe}}$ plays against a virtual adversary $\pi^{\mathrm{adv}}$ that represents disturbances and sim-to-real error. As a sufficient finite-time condition for all-time safety, the robot must reach a known controlled-invariant target set $\mathcal T$ (for example a stable stance, which a fixed policy $\pi^{\mathcal T}$ then holds) within $H$ steps without entering the failure set $\mathcal F$. With margins $g(x)\lt 0\iff x\in\mathcal F$ and $\ell(x)\ge 0\iff x\in\mathcal T$, the game value satisfies the reach–avoid Isaacs recursion (their eq. 5) $$V_k(x)=\min\Big\{g(x),\ \max\big\{\ell(x),\ \max_u\min_dV_{k+1}(f(x,u,d))\big\}\Big\},\qquad V_H(x)=\min\{\ell(x),g(x)\},$$ and the offline learning approximates a time-discounted infinite-horizon version of it. The filter is model-predictive shielding with an adversary: accept $u^{\mathrm{task}}$ at $x$ iff the imagined game (first $u^{\mathrm{task}}$, then the learned fallback, all under the learned adversary) reaches $\mathcal T$ within $H$ steps without failing (their eq. 7), else apply the fallback. Their ablation shows why the target matters: an avoid-only criterion that merely asks the imagined rollout not to fail is overly optimistic and violates safety when the horizon is short. It is a full-order filter for 36-D quadruped dynamics that transferred zero-shot to two platforms.

Latent-space reachability. Nakamura, Peters & Bajcsy, RSS 2025 run the discounted safety Bellman equation inside a generative world model trained on RGB observations (encoder $z_t\sim E_\psi(z_t|\hat z_t,o_t)$, transition $\hat z_{t+1}\sim p_\phi(\cdot|z_t,a_t)$), with the margin $\ell_\mu(z)$ given by a failure classifier over latents (so "do not spill the bag" needs no signed distance function): $$V_{\mathrm{latent}}(z)=(1-\gamma)\,\ell_\mu(z)+\gamma\min\Big\{\ell_\mu(z),\ \max_{a\in A}\mathbb E_{\hat z'\sim p_\phi(\cdot|z,a)}\big[V_{\mathrm{latent}}(\hat z')\big]\Big\},\qquad\gamma\in[0,1),$$ and filter (their eq. 7) $a^{\mathrm{exec}}_t=\pi^{\mathrm{task}}(z_t)$ if $V(\hat z_{t+1})>\epsilon$, else $\pi^{\mathrm{fb}}_{\mathrm{latent}}(z_t)$, where $\hat z_{t+1}$ is the world model's prediction under the task action and $\epsilon>0$ absorbs numerical error and latency ($\epsilon=0.4$ in simulation, $0.3$ on the Franka hardware). Nothing here is certified: the value is only as good as the world model and the classifier.

Generative policies. Filtering a diffusion policy raises a distribution-shift problem: a corrected action chunk is off the policy's data manifold. SafeDiffuser (Xiao et al., ICLR 2025) treats the denoising process itself as a controlled dynamical system and enforces CBF-type constraints by a QP at every denoising step (robust, relaxed and time-varying variants), whose finite-time diffusion invariance conclusion requires the denoising dynamics and feasibility/regularity hypotheses stated below; the time-varying version certifies final waypoint values $b\ge0$. PACS (Römer et al., ICRA 2026) instead never touches the geometric path: it only rescales time along the generated trajectory ($q(s)$ with $\dot s\ge 0$, a reparametrization of time; see Primer B), brakes as the fail-safe, and verifies a chunk by set-based reachability analysis of the robot along the path plus its braking manoeuvre against the (dynamic) obstacles, so the executed motion stays close to the training distribution while carrying a formal guarantee (three real-world human-robot interaction tasks, up to 68% higher task success than reactive CBF-type filtering; formulas schematic).

UPSi: a predictive safety filter on a probabilistic ensemble

The predictive filter of Section 7 needs a model with rigorous error bounds. Frauenknecht, Kesper, Mayfrank, Hose & Trimpe, RLC 2026 supply them for the probabilistic ensembles (PE) used in Dyna-style model-based RL: $E$ Gaussian networks $\mathcal N(\mu_{\theta_e}(s,a),\Sigma_{\theta_e}(s,a))$ (see Primer C) as estimators of an environment $s_{t+1}=\mu(s_t,a_t)+L(s_t,a_t)w_t$ whose zero-mean process noise is bounded, $\|w_t\|_2^2\le\epsilon$, with diagonal covariance $\Sigma=LL^\top$ (here $\mu$ is the nominal dynamics, not the QP multiplier).

The terminal set grows between episodes by the sampled $N$-step controllable set of the previous one (their eq. 12), so the filter becomes less conservative as data accumulates, while the guarantee (persistent feasibility of the tube-constrained problem) is inherited from robust MPC. Two honest caveats, both from the paper: the guarantees hold only under the assumptions of Theorem 1 and of the persistent-feasibility theorem below, and the benchmark evaluation uses "practical simplifications" (a naive Gaussian tube, $\ell_G=0$, no ancillary controller) that, in the authors' words, break the formal guarantees.

The companion work Dyna-SAuR (Eisele, Frauenknecht, Solowjow & Trimpe, 2026) learns the filter instead of solving it: with the certain set $\mathcal E=\{(s,a): H(\hat S_{t+1})\le\lambda_1\}$ (predictive entropy of the ensemble; see Primer C) it defines the finite-horizon certain viability kernel $$S_V(\mathcal E)=\Big\{s:\ \exists\pi\ \ \mathbb P^\pi\big[\forall t\le T,\ (\hat S_t,A_t)\in S_S(\mathcal E)\ \big|\ S_0=s\big]\ge 1-\delta\Big\},$$ and a filter policy $\mu$ that outputs a state-dependent hyperplane, $a_t=\operatorname{arg\,min}_{a\in A}\|a-\pi(s_t)\|^2$ s.t. $w_t^\top a\ge b_t$ with $(w_t,b_t)=h(\mu(s_t))$: one affine constraint, so the closed form of Section 3 applies verbatim with $b=w_t$ and $a=-b_t$ (box constraints aside). Failures during training drop by at least two orders of magnitude against the baselines on goal-reaching CartPole and MuJoCo Walker. This is viability theory (Module 7) meeting the safety-filter architecture of Section 8.

Walkthrough: From the CBF Condition to the Filtered Input

The central computation of this module: solve the CBF-QP in closed form. Every step uses only the KKT conditions of Module 2. The result is what the explorer below evaluates thousands of times per run.

Interactive: CBF-QP Safety Filter on a Robot

Run the robot simulation and compare its nominal path with the filtered path around the obstacle. Toggle actuator saturation or increase the drift to see when the desired correction cannot be applied. Read the minimum barrier value alongside the plot; the full model and safety argument are below.

Exercises

Graded practice

Each level has four problems. Write your steps before opening the answer; use the hint when you need a first move. Close the answer and retry after checking it.

Easy practice

Medium practice

Hard practice

Original longer exercises

Key Papers

Also cited above: Blanchini, Automatica 1999; Prajna & Jadbabaie, HSCC 2004; Xu, Tabuada, Grizzle & Ames, IFAC ADHS 2015; Ames, Grizzle & Tabuada, CDC 2014; Nguyen & Sreenath, ACC 2016; Reis, Aguiar & Tabuada, L-CSS 2021; Dalal et al., 2018; Taylor et al., L4DC 2020; Robey et al., CDC 2020; Choi et al., CDC 2021; DeepReach, ICRA 2021; Wabersich & Zeilinger, TAC 2023; So et al., ICRA 2024; GCBF+, T-RO 2025; Gameplay Filters, CoRL 2024; Latent Safety Filters, RSS 2025; SafeDiffuser, ICLR 2025; PACS, ICRA 2026; Dyna-SAuR, 2026; Krasowski et al., TMLR 2023; Mayne, Seron & Raković, Automatica 2005; Aswani et al., Automatica 2013; Koller et al., CDC 2018; Hewing et al., Annu. Rev. 2020.

Flashcards