14. Neural Networks in the Loop: QCs, IQCs & Dissipativity
Stability of NN-controlled systems via quadratic constraints, Zames–Falb multipliers, offset-free tracking, dissipative RNNs, synthesis with guarantees
This module assumes:
- Quadratic constraints, the S-procedure, and sufficient certificates (Module 2)
- Discrete-time Lyapunov stability, invariant sublevel sets, and regions of attraction (Primer D)
- LMIs, semidefinite programs, feasibility, and numerical margins (Module 2)
- Schur complements and ellipsoid containment (Module 2)
- Sector and slope QCs; why coupled LipSDP-Network multipliers can fail incrementally (Module 12)
- Dissipativity, storage functions, supply rates, and the KYP lemma (Module 2)
- Transfer functions, signal energy, frequency response, and FIR filters (Primer D)
- State-space realizations, controllability, and observability (Primer D)
- Recurrent equilibrium networks, well-posedness, and contraction (Module 13)
- Positive definite matrices, weighted norms, and ellipsoids (Primer A)
Book contents · Apply this chapter to a real decision · Glossary
On a first pass, follow the links below and solve the Easy set before opening the research exercises. Try each question on paper; open its hint only when stuck, then compare your reasoning with the worked solution. Use Medium questions to connect the algebra and Hard questions to check the assumptions.
§1: equilibrium and shifted signals · §2: local certificate and invariance · §3: pointwise versus summed constraints · §5: storage and gain
Practice: 4 Easy · 4 Medium · 4 Hard · original research exercises. Check readiness or use the course study guide.
Read one closed-loop energy calculation first
For the scalar loop $x^+=0.8x$, choose $V=x^2$. Substitution gives $V(x^+)-V(x)=(0.8^2-1)x^2=-0.36x^2$. This proves a decrease at every nonzero state. With a neural controller, the expression also contains activation variables $w$; a QC lets us prove the decrease without solving for every possible $w$ separately.
Track three questions alongside every theorem: around which equilibrium are the differences taken, on which state/pre-activation set is the QC valid, and at which times is it nonnegative? A local sector needs an invariant set inside its domain. A hard IQC may be nonnegative only after summing across time. Review Lyapunov sublevel sets, quadratic-constraint signs, and induction as needed.
A reinforcement-learning agent (see Primer E) or an imitation of an expensive MPC (see Primer D) hands you a neural network $\pi$ that seems to control the plant well in simulation. Before it runs on hardware you want a certificate: a proof that the closed loop is stable, together with a region of initial states from which this is guaranteed. The network is a composition of affine maps and scalar activations (see Primer E), and every common activation obeys simple quadratic inequalities (sector and slope bounds). Pulling all activations into one diagonal block turns the closed loop into a Lur'e system, a linear system in feedback with a static nonlinearity, and sixty years of robust control apply: the S-procedure, integral quadratic constraints (IQCs), dissipativity and multipliers. This page follows the line that Patricia Pauli and co-authors built on the template of Yin, Seiler and Arcak (local sector constraints, then Zames–Falb multipliers, offset-free tracking and recurrent networks, for which see Primer E), then turns to synthesis, to reachability and to verified neural Lyapunov functions, and ends with the 2025–2026 work on scale. The tools are those of Module 2; the network-level constraints are those of Module 12.
1. The Feedback Setup and Loop Transformations
Start with the plainest object: a linear time-invariant plant (see Primer D) and a feedforward network used as a state-feedback controller. (Output feedback $u=\pi(Cx)$ only changes the first weight to $W_0C$; Yin et al., Remark 2.)
The first move, used by Fazlyab, Morari & Pappas (DeepSDP) for a network alone and by Yin, Seiler & Arcak, TAC 2022 for the loop, is to separate what is linear from what is not.
All pre-activations $v$ leave the linear block, all activations $w$ return to it; the state and the control stay inside the linear block.
Why this form. Every certificate on this page has the same shape: a quadratic Lyapunov or storage function for the linear block, plus quadratic constraints that everything in the $\phi$ block is known to satisfy, glued together by the S-procedure. Because $N_{vw}$ is strictly block lower triangular, $w$ can be computed layer by layer, so the loop is well posed. Implicit (equilibrium) networks and RENs allow a full $N_{vw}$; then well-posedness needs a condition such as $2T-TN_{vw}-N_{vw}^\top T\succ0$ for a diagonal $T\succ0$ and activations slope-restricted in $[0,1]$ (Revay, Wang & Manchester, TAC 2024, eq. 12).
Step 1 (equilibrium equations). An equilibrium satisfies $x_*=Ax_*+Bu_*$, $\begin{bmatrix}u_*\\ v_*\end{bmatrix}=N\begin{bmatrix}x_*\\ w_*\\ 1\end{bmatrix}$, $w_*=\phi(v_*)$ (Yin et al., eq. 12). Finding one is a fixed-point problem $x_*=Ax_*+B\pi(x_*)$; for a bias-free network whose activation satisfies $\varphi(0)=0$ (tanh, ReLU), $x_*=0$ always works, with $v_*=w_*=0$. Bias-free is not enough on its own: a sigmoid has $\sigma(0)=1/2$, so $\pi(0)\ne0$ in general.
Step 2 (subtract). Subtracting the equilibrium equations from the loop equations, the constant column $N_{\cdot b}$ multiplies $1-1=0$:
with the shifted activations $\tilde\varphi_i(s)=\varphi(s+v_{*,i})-\varphi(v_{*,i})$.
Step 3 (what survives). (i) $\tilde\varphi_i(0)=0$, so the origin is the equilibrium of the shifted loop. (ii) Every chord of $\tilde\varphi_i$ is a chord of $\varphi$, so slope bounds are inherited unchanged. (iii) What is lost: if $v_{*,i}\ne v_{*,j}$ then $\tilde\varphi_i\ne\tilde\varphi_j$. The stacked map is then no longer a repeated nonlinearity (the same scalar function in every channel), and multipliers that exploit a repeated nonlinearity (coupling different neurons through monotonicity, such as ZF-R) become invalid between such neurons (Section 3, Exercise 14.6); multipliers that use only each neuron's own sector or slope bounds, including the full-block circle multipliers of Section 3, stay valid. Pauli et al. (CDC 2021) make exactly this point: the shift "eliminates the bias terms", but "in general" the components "are not identical", so they use repeated-nonlinearity IQCs only for bias-free networks at $x_*=0$. That case (with $\varphi(0)=0$, so that every $v_{*,i}=0$) is sufficient, not necessary: coupling stays valid within any group of neurons that carry the same activation with equal $v_{*,i}$ and share their slope bounds (Section 3). A general equilibrium shift destroys this.
- $\varphi$ satisfies the (offset) sector $[\alpha,\beta]$ around $(v_*,\varphi(v_*))$ on $\mathcal I$ if $(\Delta\varphi-\alpha\,\Delta v)(\beta\,\Delta v-\Delta\varphi)\ge0$ for all $v\in\mathcal I$ (Yin et al., Defs. 2–4). Global: $\mathcal I=\mathbb R$; local: a bounded $\mathcal I$; centred: $v_*=0$, $\varphi(0)=0$.
- $\varphi$ is slope-restricted in $[\mu,\nu]$ on $\mathcal I$ if $\mu\le\frac{\varphi(v)-\varphi(v')}{v-v'}\le\nu$ for all $v\ne v'$ in $\mathcal I$.
tanh and ReLU are globally slope-restricted, hence sector-bounded, in $[0,1]$. On $|v|\le\bar v$ the centred tanh has the local sector $[\tanh(\bar v)/\bar v,\ 1]$ and the local slope bounds $[1-\tanh^2\bar v,\ 1]$ (Exercise 14.2).
tanh is almost the identity, and a local sector $[\alpha,1]$ with $\alpha\gt0$ excludes the switched-off controller. The price is that the constraint is valid only while the pre-activations stay in a box, so the certificate must also prove that they do. For the lower sector bound use the chord slope $\tanh(\bar v)/\bar v$, not the smallest slope $1-\tanh^2\bar v$: the latter is the right lower bound for slope restriction and a valid but loose sector bound (for $\bar v=2$: $0.482$ against $0.071$; Module 2 compares the two).A relation is a set of pairs $(a,b)$; unlike a function it may assign several $b$ values to one $a$. It is monotone when $(a-a')(b-b')\ge0$ for every two of its pairs. Here $a=\nu v-w$ and $b=w-\mu v$: a chord of $\varphi$ with slope $s\in[\mu,\nu]$ gives $\Delta a=(\nu-s)\Delta v$, $\Delta b=(s-\mu)\Delta v$, so $(\Delta a)(\Delta b)=(\nu-s)(s-\mu)(\Delta v)^2\ge0$. If $s=\nu$ on some interval, then $\Delta a=0$ while $\Delta b\ne0$: one value of $a$ carries several values of $b$, which is why the transformed object is a relation rather than a function. The monotonicity arguments of Section 3 use only such product inequalities (and $ab\ge0$ through the origin), so multivaluedness does no harm.
The Lur'e problem asks for stability of a linear system $G$ in feedback with any nonlinearity from a class (a sector, or a slope restriction). Two classical warnings shape everything below. First, checking the linear gains in the sector is not enough in general (Aizerman's conjecture is false; see Primer D); certificates must use the constraint itself. Second, the answer depends on the class: sector-bounded, slope-restricted and slope-restricted and odd nonlinearities admit progressively richer multipliers.
Static multipliers (a constant $\lambda\ge0$ times the sector QC, the S-procedure) give circle-criterion type tests; by the KYP lemma (Module 2, Rantzer 1996) the frequency-domain and LMI forms are equivalent (the strict versions without any controllability assumption). Dynamic multipliers exploit that the nonlinearity has no memory and bounded slopes. The continuous-time Zames–Falb theorem, in the form collected by Carrasco, Turner & Heath, EJC 2016 (Thm. 1): let $G$ be a stable, proper, real-rational scalar transfer function in negative feedback $v=f-Gw$, $w=\varphi(v)+e$ with zero initial state, let $\varphi(0)=0$ with every chord slope in $[0,k]$, $0\lt k\lt\infty$, and assume that the loop is well posed (unique causal solutions for every input of finite energy on each finite interval). If $M(j\omega)=m_0-H(j\omega)$, where $H$ has a possibly two-sided, integrable impulse response $h$ (see Primer D) with $\|h\|_1\lt m_0$ and $h\ge0$ unless $\varphi$ is odd, and there is an $\varepsilon\gt0$ with $$\mathrm{Re}\big\{M(j\omega)\,(1+kG(j\omega))\big\}\ge\varepsilon\quad\forall\omega\in\mathbb R,$$ then the loop is $L_2$-stable: there is a $c\lt\infty$ with $\|(v,w)\|_{L_2[0,T]}\le c\,\|(f,e)\|_{L_2[0,T]}$ for every horizon $T$, a bound from input energy to signal energy, not a region-of-attraction statement. With $H=0$ this is the circle-type test. Both qualifications matter. For $G(s)=-s/(s+1)=-1+1/(s+1)$, $M=1$ and $k=1$ one has $\mathrm{Re}\{1+G(j\omega)\}=1/(1+\omega^2)\gt0$ at every frequency, but not uniformly. With $\varphi(v)=v$ that loop is not well posed: the direct feedthrough $G(\infty)=-1$ makes the coefficient of the instantaneous (algebraic) loop $1+G(\infty)=0$. And $(1+G)^{-1}=s+1$ has a frequency response that grows without bound, so it has no finite $L_2$ gain. Sections 2–3 use discrete-time, time-domain versions of both tests.
2. Local Sector QCs and the Stability LMI (Yin, Seiler, Arcak)
Yin, Seiler & Arcak (TAC 67(4):1980–1987, 2022; arXiv 2020) gave the template that the rest of this page extends: a quadratic Lyapunov function, local sector constraints on the activations obtained by interval bound propagation, and one SDP whose solution is an ellipsoidal inner approximation of the ROA. Their Theorem 2 adds plant perturbations described by IQCs.
The bounds $[\underline v,\bar v]$ must be computed, and they must contain every pre-activation the closed loop can produce. Yin et al. choose a box for the first layer and push it through the network with interval bound propagation (Gowal et al., IBP):
- Choose a first-layer box $[\underline v^1,\bar v^1]$ symmetric about $v^1_*$, e.g. $\bar v^1-v^1_*=v^1_*-\underline v^1=\delta\,\mathbf 1$.
- For $i=1,\dots,\ell-1$: // monotone activation
- output box $[\underline w^i,\bar w^i]=[\phi(\underline v^i),\phi(\bar v^i)]$; centre $c=\tfrac12(\bar w^i+\underline w^i)$, radius $r=\tfrac12(\bar w^i-\underline w^i)$;
- $\bar v^{i+1}_j=(W_i)_jc+(b_i)_j+\sum_k|(W_i)_{jk}|r_k$, $\ \underline v^{i+1}_j=(W_i)_jc+(b_i)_j-\sum_k|(W_i)_{jk}|r_k$ // exact max/min of an affine function over a box
- For each neuron compute a valid offset sector on its interval. For
tanh: $\beta_i=1$ and $\alpha_i=\min\Big\{\frac{\tanh\bar v_i-\tanh v_{*,i}}{\bar v_i-v_{*,i}},\ \frac{\tanh v_{*,i}-\tanh\underline v_i}{v_{*,i}-\underline v_i}\Big\}$ (Yin et al., Sec. II-C; $\beta_i$ can be tightened further).
Why the interval step is exact, and where it loses. For a row $a=(W_i)_j$ write each output as $w_k=c_k+e_k$ with $|e_k|\le r_k$. Then $aw=ac+\sum_ka_ke_k$ is largest when every $e_k=r_k\,\mathrm{sign}(a_k)$, which gives $ac+\sum_k|a_k|r_k$ (smallest with the opposite signs). The $e_k$ can be chosen independently only because the set is a box; the true outputs of a layer are correlated, so the box contains combinations the network never produces, and the overestimate compounds layer by layer. In the last step $\beta_i=1$ is a valid, not always the tightest, upper bound; if an endpoint coincides with $v_{*,i}$, its chord ratio is read as the limit $\tanh'(v_{*,i})$, and a degenerate interval $\{v_{*,i}\}$ satisfies the sector QC trivially.
Statement. (a) $x_*$ is locally asymptotically stable (the paper says "locally stable"; the proof gives asymptotic stability) and (b) $\mathcal E(P,x_*)$ is an inner approximation of the ROA.
In words. On the ellipsoid, every first-layer pre-activation stays in its box, so every activation obeys its local sector; the first LMI says that $V$ strictly decreases for every nonlinearity obeying those sectors.
Why each assumption matters. The equilibrium: the QCs are centred at it, and without one there is nothing to be stable. The box and Property 1: the sector bounds are only true inside the box; a box that misses reachable pre-activations makes the certificate unsound. The second LMI: it is what keeps the state inside the box (below), without it the local sectors may be used where they are false (Exercise 14.4). Strictness of the first LMI: it yields a margin $\varepsilon\|x-x_*\|^2$ and hence convergence, not just boundedness.
Step A (ellipsoid inside the box). The Schur complement of the second LMI with respect to $P\succ0$ gives $(W_0)_iP^{-1}(W_0)_i^{\top}\le(\bar v^1_i-v^1_{*,i})^2$. For $x\in\mathcal E(P,x_*)$ put $y=x-x_*$, $y^{\top}Py\le1$. By Cauchy–Schwarz in the $P$ inner product,
Since $v^1-v^1_*=W_0(x-x_*)$, every $x\in\mathcal E$ produces $v^1\in[\underline v^1,\bar v^1]$, and by Algorithm 1 every later pre-activation is in its box. So on $\mathcal E$ the QC of the Lemma holds. (Yin et al. cite a lemma of Hindi and Boyd for this containment; the Cauchy–Schwarz line above is its proof, and the bound is attained, so the condition is also necessary.)
Step B (the quadratic form is the Lyapunov difference plus the QC). Strictness gives $\varepsilon\gt0$ with $\mathcal M\preceq-\varepsilon I$. Multiply by $\zeta_t=(\tilde x_t,\tilde w_t)$ on both sides. Because $R_V\zeta_t=(\tilde x_t,\tilde u_t)$, $R_\phi\zeta_t=(\tilde v_t,\tilde w_t)$ and $\tilde x_{t+1}=A\tilde x_t+B\tilde u_t$:
Step C (induction removes the circularity; proof by induction: Primer 0). If $x_t\in\mathcal E$, Step A makes the QC valid at time $t$, so $V(x_{t+1})\le V(x_t)-\varepsilon\|x_t-x_*\|^2\le1$, i.e. $x_{t+1}\in\mathcal E$. Only the QC at the current time is used, so there is no circular argument: invariance is proved one step at a time.
Step D (convergence). Summing, $\varepsilon\sum_t\|x_t-x_*\|^2\le V(x_0)\le1$. The terms of a convergent series tend to zero (Primer 0): otherwise $\|x_t-x_*\|\ge a$ for some $a\gt0$ and infinitely many $t$, and each such $t$ would add at least $\varepsilon a^2$ to the sum. So $x_t\to x_*$. Stability in the sense of Lyapunov (starting close means staying close) comes from the invariant sublevel sets $\{V\le c\}$, $c\le1$, and $\lambda_{\min}(P)\|x-x_*\|^2\le V(x)\le\lambda_{\max}(P)\|x-x_*\|^2$; with $V$ a strict Lyapunov function on the invariant set $\mathcal E$ this is local asymptotic stability (Khalil, Thm. 4.1, as cited by Yin et al.). $\blacksquare$
The containment LMI checks where the local activation model is allowed: every state in the proposed ellipsoid must place the first pre-activation inside its chosen box. The decrease LMI then checks what happens next from any such state. Starting in the ellipsoid, one step lowers $V$, keeps the state inside the same ellipsoid, and permits the argument again.
This order matters. First establish containment at the current state, then decrease, then invariance by induction. A simulation that stays inside the box is useful evidence about that trajectory, but it cannot replace containment for every state in the certified set. The Easy ellipsoid question and Hard invariance question let you practice these jobs separately.
Computing a large certified ellipsoid. For a fixed box both LMIs are linear in $(P,\lambda)$, and Yin et al. solve
A small $\mathrm{trace}(P)=\sum_j1/r_j^2$ (with $r_j=1/\sqrt{\lambda_j(P)}$ the semi-axes) means a large ellipsoid. It is only a surrogate: it favours long axes but neither maximises the volume nor guarantees that the result contains a competing ellipsoid. The volume $\propto\det(P)^{-1/2}$ would be more natural, but minimising $\log\det P$ is minimising a concave function, whereas the trace is linear. The box size is not a convex variable: shrinking $\delta$ tightens the sectors ($\alpha$ grows) but confines the ellipsoid to a smaller box, enlarging it loosens the sectors until the first LMI becomes infeasible (their Remark 1). Yin et al. grid $\delta$; Pauli et al. (CDC 2021) bisect for the largest feasible $\delta$ and then run a golden-section search on $\mathrm{trace}(P)$. The explorer below does the same: it brackets $\delta\in[0.02,40]$, takes 18 bisection steps at the geometric midpoint $\sqrt{\delta_{lo}\delta_{hi}}$ (bisection in $\log\delta$), then 14 golden-section steps for $\mathrm{trace}(P)$ on $[0.05\,\delta_{\max},\,0.998\,\delta_{\max}]$. These are stopping rules: they locate the feasibility boundary and the best box only approximately.
Bisection keeps an interval $[\delta_{lo},\delta_{hi}]$ with $\delta_{lo}$ feasible and $\delta_{hi}$ infeasible, tests the midpoint and keeps the half where the switch happens, so the interval halves at every step. It finds the boundary only if feasibility is nested (every box smaller than a feasible one is feasible) and every feasibility test is reliable; here that is an assumption, not a proved property. Golden-section search minimises a function of one variable on an interval by comparing its values at two interior points and discarding the part beyond the worse one; the interval shrinks by the factor $0.618$ per step, and the method is guaranteed to approach the minimiser only if the function is unimodal (decreasing, then increasing). Unimodality of $\delta\mapsto\min\mathrm{trace}(P)$ has not been shown, so the outer search is a heuristic. Whatever box it returns, the certificate for that box is valid.
The extended system. Work in deviations (tildes dropped) and keep $s=(w,q)$ as the free signals. Substituting $u=N_{ux}x+N_{uw}w$ gives $x^+=A_cx+B_ss$ and $p=C_cx+D_ss$ with $A_c=A+B_2N_{ux}$, $B_s=[B_2N_{uw}\ \ B_1]$, $C_c=C+D_2N_{ux}$, $D_s=[D_2N_{uw}\ \ D_1]$. Write the filter as $\psi^+=A_f\psi+B_{fp}p+B_{fq}q$, $r=C_f\psi+D_{fp}p+D_{fq}q$, and let $J_w=[I\ 0]$, $J_q=[0\ I]$ pick $w$ and $q$ out of $s$. Substituting $p$ into the filter gives $\zeta^+=A_e\zeta+B_es$, $r=C_e\zeta+D_es$ and $(v,w)=F_\phi(\zeta,s)$ with
The last inequality has size $n_x+n_\psi+n_\phi+n_q$ ($n_\psi$ filter states, $n_q$ uncertainty channels); the containment LMIs are those of Theorem 1 with the row $[(W_0)_j\ \ 0]$. Multiplying by $(\zeta_t,s_t)$ gives $V(\zeta_{t+1})-V(\zeta_t)+r_t^{\top}M_\Delta r_t+(\text{sector QC})\le-\varepsilon\|(\zeta_t,s_t)\|^2$. The IQC term is nonnegative only as a sum, so one sums over $t$ exactly as in the hard-IQC proof of Section 3 (Steps 2–4 there, including the extension argument for the local sectors); with $\psi_0=0$ this keeps $\zeta_t$ in the ellipsoid and drives it to zero.
The off-by-one IQC for one neuron. With local slope bounds $[\mu,\nu]$ put $a_t=\nu v_t-w_t$ and $b_t=w_t-\mu v_t$, a monotone relation through the origin (Section 1). Yin et al. (Sec. III-D) take the filter state $\psi_{t+1}=w_t-\nu v_t=-a_t$ with $\psi_0=0$, the output $r_t=(a_t-a_{t-1},\ b_t)$ and the middle matrix $\begin{bmatrix}0&\eta\\ \eta&0\end{bmatrix}$, $\eta\ge0$, so $r_t^{\top}Mr_t=2\eta\,b_t(a_t-a_{t-1})$ (with $a_{-1}=0$). Over a horizon $\sum_tb_t(a_t-a_{t-1})=\mathbf b^{\top}H\mathbf a$, where $H$ has $1$ on the diagonal and $-1$ on the subdiagonal: off-diagonal entries $\le0$ and all row and column sums $\ge0$, so the rearrangement lemma of Section 3 gives $\mathbf b^{\top}H\mathbf a\ge0$ for every $N$. The terms themselves can be negative: for $\varphi(v)=v/2$, $[\mu,\nu]=[0,1]$ and $(v_0,v_1)=(2,1)$ they are $1$ and $-1/4$, the partial sums $1$ and $3/4$. That is exactly the difference between a hard IQC and a pointwise QC. Summing such terms over the neurons with weights $\eta_i\ge0$ gives the $n_\phi$-state filter of the theorem.
Examples in the paper. (1) An inverted pendulum ($m=0.15$ kg, $l=0.5$ m, friction $0.5$ Nms/rad, input saturation at $0.7$ Nm, i.e. clipping to $[-0.7,0.7]$, a projection onto a box as in Primer B) with a policy-gradient controller: two tanh layers of 32 neurons, biases fixed to zero during training so that $x_*=0$, sampling time $0.02$ s. The term $\theta-\sin\theta$ is treated as a perturbation that is slope-restricted in $[0,0.2548]$ and sector-bounded in $[0,0.087]$ for $|\theta|\le0.73$, the saturation by a local sector, and the first-layer box uses $\delta=0.1$; both assumptions are checked on the resulting ellipsoid. (2) Vehicle lateral dynamics with a 32–32 tanh policy, saturation at $\pi/6$ and a norm-bounded LTI actuator uncertainty ($\|\Delta\|_\infty\le0.1$): adding the off-by-one IQC reduces $\mathrm{trace}(P_x)$ from $4.4$ to $2.9$, increases $\det(P_x^{-1})$ (proportional to the squared volume) from $3.2\times10^5$ to $1.1\times10^6$, and raises the largest feasible box from $\delta=0.67$ to $\delta=1.4$.
Complexity. The first LMI has size $n_x+n_\phi$ and, with a diagonal $T$, about $n_x(n_x+1)/2+n_\phi$ variables; the $n_1$ containment LMIs have size $1+n_x$. By the usual interior-point estimate (each Newton step costs on the order of $mn^3+m^2n^2$ floating-point operations, or flops, for $m$ variables and matrix size $n$, and about $\sqrt n$ steps are needed, times a factor that grows with the logarithm of the requested accuracy) a wide network costs roughly $n_\phi^4$ per step and $n_\phi^{4.5}$ in total, up to constants ($O(\cdot)$ notation: Primer 0; doubling $n_\phi$ multiplies $n_\phi^4$ by $16$): comfortable for hundreds of neurons, hopeless for millions, which motivates Section 8.
3. Dynamic Multipliers: Acausal Zames–Falb (Pauli et al.)
A static QC uses one time instant. But an activation is memoryless and slope-restricted, so values at different times are related too: $w_t-w_{t-1}=\tilde\varphi(v_t)-\tilde\varphi(v_{t-1})$ has the slope of a chord. Pauli, Gramlich, Berberich & Allgöwer, CDC 2021 brought the full class of dynamic multipliers for slope-restricted nonlinearities to NN loops: full-block circle and Yakubovich-type multipliers (following Fetzer and Scherer) combined with acausal Zames–Falb multipliers realised by FIR filters (see Primer D).
The symbols. $\ell_2^n$ is the set of sequences $(v_t)_{t\ge0}$ in $\mathbb R^n$ with finite energy $\sum_t\|v_t\|^2\lt\infty$; $\hat v(e^{j\omega})=\sum_{t\ge0}v_te^{-j\omega t}$ is the discrete-time Fourier transform (Primer D); ${}^{*}$ is the conjugate transpose, and $\Pi(e^{j\omega})$ is Hermitian, so the integrand is real. With $\Pi=\Psi^{*}M\Psi$, Parseval's theorem turns the integral into $2\pi\sum_{t\ge0}r_t^{\top}Mr_t$, which is why the soft IQC is the same condition written in time. The hard IQC is applied to finite prefixes of a trajectory, so the stability proofs below never assume in advance that the closed-loop signals have finite energy.
The odd case. $\varphi$ is odd if $\varphi(-s)=-\varphi(s)$. Then the sign conditions are replaced by: for every channel $i$ (writing $m_{j,ik}$ for the entries of $M_j$) $$\begin{gathered}m_{0,ii}\ge\sum_{k\ne i}|m_{0,ik}|+\sum_{j\ne0}\sum_{k}|m_{j,ik}|,\\ m_{0,ii}\ge\sum_{k\ne i}|m_{0,ki}|+\sum_{j\ne0}\sum_{k}|m_{j,ki}|,\end{gathered}$$ i.e. each diagonal entry of $M_0$ dominates the absolute sum of all other entries in its row, and in its column, over all channels and delays; off-diagonal entries of either sign are then allowed. (Pauli et al. print the second sum over all $j$; with $j=0$ included the entries of $M_0$ would be counted twice, so we read it as $j\ne0$.) The loop-transformed channels of an odd $\varphi$ are odd again, but a shifted activation $\tilde\varphi(s)=\tanh(v_*+s)-\tanh(v_*)$ with $v_*\ne0$ is not odd, so this larger class is available only for equilibria at $v_*=0$ (e.g. bias-free networks with $x_*=0$).
Written out, the quadratic form is $2w_t^{\top}(M_0v_t+M_{-1}v_{t-1}+\dots+M_{-\ell}v_{t-\ell})+2\sum_{j=1}^{\ell}w_{t-j}^{\top}M_jv_t$. The multiplier $\sum_jM_jz^{-j}$ contains positive powers of $z$ (future values), which is what acausal means; the factorisation nevertheless uses only delays, because the future terms are moved to the other signal: "$v_t$ against a past $w$" plays the role of "$w$ against a future $v$". The IQC is a property of the signals, not a filter one implements, so non-causality costs nothing.
In words: over every finite horizon, the multiplier-weighted products of current and delayed inputs and outputs of the nonlinearity add up to something nonnegative. Why the assumptions: slope restriction (not only a sector) is what makes products at different times valid; the repeated structure is what allows full $M_j$ to couple different neurons (derivation below, Exercise 14.6), and after the loop transformation the channels are repeated only if they share $(\mu,\nu)$. The paper writes the transformation with per-neuron vectors $\mu,\nu$; with coupled $M_j$ that is not enough: for $\varphi(s)=s/2$ in two channels, $\mu=(1/4,0)$, $\nu=(1,3/4)$ and $M_0=\begin{bmatrix}1 & -1\\ -1 & 1\end{bmatrix}$ (all other $M_j=0$), the pair $v=(1,1)$, $w=(1/2,1/2)$ gives the value $-1/8$ at a single time step. The zero values before $t=0$ (zero initial filter state) give the finite-horizon, hard version that the Lyapunov argument needs.
A matrix is doubly hyperdominant if its off-diagonal entries are nonpositive and its row and column sums are nonnegative. The key lemma (Willems and Brockett, as used by Pauli et al., Lemma 2): if $\varphi$ is monotone ($\mathrm{slope}[0,\infty]$) with $\varphi(0)=0$ and $\mathsf M$ is doubly hyperdominant, the repeated map $\phi$ satisfies $v^{\top}\mathsf M\phi(v)\ge0$ for all $v$.
The $2\times2$ case shows both ingredients. Take $\mathsf M=\begin{bmatrix}m & -c\\ -c & m\end{bmatrix}$ with $m\ge c\ge0$. Then
The first bracket is $\ge0$ by the sector property ($v\varphi(v)\ge0$); the second product is $\ge0$ by monotonicity of one and the same function evaluated at $v_1$ and $v_2$ (a rearrangement inequality). If the two channels carried different functions $\varphi_1\ne\varphi_2$, the second term would read $(v_1-v_2)(\varphi_1(v_1)-\varphi_2(v_2))$, which can be negative: that is why this monotonicity-based coupling of different neurons needs a repeated nonlinearity, while coupling the same neuron at different times (the off-diagonal time structure of $\tilde P$) is always fine for a time-invariant activation.
Hard IQC. Stack the finite sequence $(v_0,\dots,v_N)$ and $(w_0,\dots,w_N)$. The sum over $t$ of the FIR quadratic form equals $\mathbf v^{\top}\mathsf M_N\mathbf w$ for a banded block-Toeplitz matrix $\mathsf M_N$ (constant along its block diagonals, Primer A) built from the $M_j$; conditions (row/column sums and signs) make $\mathsf M_N$ doubly hyperdominant for every $N$, and the lemma applied to the long vector gives nonnegativity for every horizon, i.e. a hard IQC (Pauli et al., Appendix B).
Example: one channel, one delay. Summing the written-out form over $t=0,1,2$ (with $v_{-1}=w_{-1}=0$) gives $2\mathbf w^{\top}H\mathbf v$ with $H=\begin{bmatrix}m_0&m_1&0\\ m_{-1}&m_0&m_1\\ 0&m_{-1}&m_0\end{bmatrix}$, so $\mathsf M_N=2H^{\top}$. Every interior row and column contains $m_{-1}+m_0+m_1\ge0$; the first and last ones miss an off-diagonal entry, which is $\le0$, so their sums are even larger. Truncating to a finite horizon therefore keeps $\mathsf M_N$ doubly hyperdominant (in the odd case it only removes terms from the absolute sums). The zero padding before $t=0$ is legitimate because the shifted nonlinearity passes through zero.
Slope bounds $[\mu,\nu]$. The lemma is applied to the loop-transformed channels $a_i=\nu_iv_i-w_i$, $b_i=w_i-\mu_iv_i$. For a chord of slope $s\in[\mu_i,\nu_i]$, $\Delta a_i\,\Delta b_i=(\nu_i-s)(s-\mu_i)\Delta v_i^2\ge0$, so each $a_i\mapsto b_i$ is monotone (possibly multi-valued), and it passes through $0$ when $\varphi(0)=0$. The rearrangement step for coupled neurons $i,k$ needs these two relations to be the same. For a repeated $\varphi$ that is guaranteed when $\mu_i=\mu_k$ and $\nu_i=\nu_k$, and fails in general otherwise. Per-neuron local bounds from interval propagation usually differ, so a coupled multiplier needs one common pair of bounds, valid on the common interval of the group, as in Step 4 of the proof sketch below. The counterexample in the theorem's "why" line fails already at a single time step.
Making "for every $\Delta$" finite (a standard sufficient test; the paper does not spell out its implementation): write $\Pi=\begin{bmatrix}\Pi_{vv}&\Pi_{vw}\\ \Pi_{vw}^{\top}&\Pi_{ww}\end{bmatrix}$ and require $\Pi_{ww}\preceq0$. For fixed $v$ the form $v^{\top}[I;\Delta]^{\top}\Pi[I;\Delta]v$ is then a concave quadratic function of the diagonal entries of $\Delta$ (its Hessian is $2\,\mathrm{diag}(v)\Pi_{ww}\mathrm{diag}(v)\preceq0$), so it is smallest at a vertex of the box, and checking the LMI at the $2^n$ vertex matrices $\Delta$ (at $2^{2n}$ for $\Pi^{\rm cy}$) suffices. The number of vertices grows exponentially with $n$.
With the IQCs in hand, the stability test is a dissipation inequality for the plant plus filter states. Let $\eta=(\tilde x,\xi)$ stack the plant state and the state $\xi$ of the filters $\Psi$ driven by $(\tilde v,\tilde w)$, with realisation $\eta_{t+1}=A_{\rm tot}\eta_t+B_{\rm tot}\tilde w_t$, $r_t=C_{\rm tot}\eta_t+D_{\rm tot}\tilde w_t$.
Write the (stacked) filters as $\xi_{t+1}=A_\psi\xi_t+B_{\psi v}\tilde v_t+B_{\psi w}\tilde w_t$, $r_t=C_\psi\xi_t+D_{\psi v}\tilde v_t+D_{\psi w}\tilde w_t$, and substitute the loop equations $\tilde u=N_{ux}\tilde x+N_{uw}\tilde w$, $\tilde v=N_{vx}\tilde x+N_{vw}\tilde w$ and $\tilde x_{t+1}=A\tilde x_t+B\tilde u_t$:
For the FIR filters of this section these four matrices are fixed (shift registers); the multiplier coefficients sit only in the middle matrix $M$, which is why the LMI below is linear in $(P,M)$.
In words: a quadratic storage for the plant and filter states that dissipates against the IQC supply, plus an ellipsoid that keeps the first layer inside its box, certifies that ellipsoid. Why the assumptions: the slope and sector bounds hold only inside the box, hence the containment LMI; the multiplier class must be valid for the shifted, in general non-repeated nonlinearity (coupled $M_j$ only within groups of neurons that carry the same shifted function and share their slope bounds, e.g. a bias-free network with $\varphi(0)=0$ at $x_*=0$ and one common interval per group; diagonal $M_j$ otherwise); the hard IQC requires the filter to start at $\xi_0=0$, which is why only that slice of $\mathcal E(P,0)$ is certified.
Step 1. Multiply the first LMI by $(\eta_t,\tilde w_t)$: with $\eta_{t+1}=A_{\rm tot}\eta_t+B_{\rm tot}\tilde w_t$ and $r_t=C_{\rm tot}\eta_t+D_{\rm tot}\tilde w_t$ this is $\eta_{t+1}^{\top}P\eta_{t+1}-\eta_t^{\top}P\eta_t+r_t^{\top}Mr_t\le-\varepsilon\|\eta_t\|^2$.
Step 2 (sum, then use the hard IQC). Summing over $t=0,\dots,N$ gives $\eta_{N+1}^{\top}P\eta_{N+1}+\varepsilon\sum_t\|\eta_t\|^2\le\eta_0^{\top}P\eta_0-\sum_{t=0}^Nr_t^{\top}Mr_t\le\eta_0^{\top}P\eta_0$. With dynamic multipliers $V(\eta_t)$ need not decrease at every step; only $V(\eta_t)\le V(\eta_0)$ is guaranteed. That is enough to keep a trajectory that starts with $\xi_0=0$ and $\eta_0^{\top}P\eta_0\le1$ in the ellipsoid, and with the summable $\|\eta_t\|^2$ it gives convergence. It does not make the augmented ellipsoid invariant for restarts with a nonzero filter state: the hard IQC needs $\xi_0=0$. So the certified initial states are the slice $\{\tilde x_0:\ \tilde x_0^{\top}P_x\tilde x_0\le1\}$ of the ellipsoid at $\xi=0$, not its (larger) projection onto the $x$-coordinates.
Step 3 (box). The Schur complement of the second LMI gives $(Q_j\tilde x_t)^2\le(d^1_j)^2\eta_t^{\top}P\eta_t\le(d^1_j)^2\eta_0^{\top}P\eta_0\le(d^1_j)^2$ when $\xi_0=0$ and $\tilde x_0\in\mathcal E(P_x,0)$ (because then $\eta_0^{\top}P\eta_0=\tilde x_0^{\top}P_x\tilde x_0$).
Step 4 (local validity). A hard IQC is a statement about whole sequences, so the one-step induction of Section 2 is not available. Pauli et al. first assume the bounds hold globally and then argue that behaviour outside the box is irrelevant. One way to make this precise: replace $\tilde\varphi_i$ outside its interval $[\underline d_i,\bar d_i]$ by an affine extension with slope $\alpha_i$ (the tight lower sector bound, which lies in $[\mu_i,\nu_i]$ because it is a chord slope); every new chord slope and every new ratio $\hat\varphi_i(s)/s$ is a convex combination of old ones and $\alpha_i$, so the extension satisfies the bounds globally (for the repeated classes use one common interval, and hence common bounds, for all coupled neurons, so that the extension is again repeated); the theorem applies to it, trajectories from $\mathcal E$ never leave the box (Step 3), and on the box the extension equals the true activation, so these trajectories are trajectories of the real loop.
Why the extension keeps the bounds. Write the interval as $[\underline d,\bar d]\ni0$ and set $\hat\varphi(s)=\varphi(\bar d)+\alpha(s-\bar d)$ for $s\gt\bar d$, $\hat\varphi(s)=\varphi(\underline d)+\alpha(s-\underline d)$ for $s\lt\underline d$, and $\hat\varphi=\varphi$ in between. For $s\gt\bar d\gt0$, $\hat\varphi(s)/s=\frac{\bar d}{s}\cdot\frac{\varphi(\bar d)}{\bar d}+\big(1-\frac{\bar d}{s}\big)\alpha$, a convex combination of an old sector ratio and $\alpha$, hence in $[\alpha,\beta]$. A chord from $s_1\in[\underline d,\bar d]$ to $s_2\gt\bar d$ has slope $\frac{\bar d-s_1}{s_2-s_1}\cdot\frac{\varphi(\bar d)-\varphi(s_1)}{\bar d-s_1}+\frac{s_2-\bar d}{s_2-s_1}\,\alpha$, a convex combination of an inside chord slope and $\alpha$, hence in $[\mu,\nu]$. Chords with both ends outside on one side have slope $\alpha$, a chord across the whole interval is a convex combination of the same kind, and the left side is symmetric. $\blacksquare$
How much structure to exploit. The multiplier class is a dial between conservatism and cost (Pauli et al., Table I; $n$ neurons, FIR orders $\ell_-,\ell_+$):
| Multiplier class | Encodes | Valid for | Decision variables |
|---|---|---|---|
| Diagonal circle $\Pi^d_{[\alpha,\beta]}$ | sector | any activations | $n$ |
| Full-block circle $\Pi_{[\alpha,\beta]}$ | sector | any activations | $2n(2n+1)/2$ |
| Circle + Yakubovich $\Pi^{cy}$ | sector and slope (one-step differences) | any activations | $4n(4n+1)/2$ |
| ZF, diagonal $M_j$ | slope, across time | any activations | $(\ell_-+\ell_++1)n$ |
| ZF-RL, block-diagonal by layer | slope, across time and neurons of a layer | repeated within each layer, one slope bound per layer (e.g. bias-free, $\varphi(0)=0$, $x_*=0$) | $(\ell_-+\ell_++1)\sum_in_i^2$ |
| ZF-R, full $M_j$ | slope, across time and all neurons | repeated, one slope bound for all neurons (e.g. bias-free, $\varphi(0)=0$, $x_*=0$) | $(\ell_-+\ell_++1)n^2$ |
Numbers from the paper. (1) A linearised inverted pendulum (see Primer D) with a bias-free $5$–$5$ tanh network trained on MPC data ($|u|\le1$): diagonal ZF ($\ell=1$) barely improves on the diagonal circle criterion, while the repeated classes ZF-RL and ZF-R give markedly smaller $\mathrm{trace}(P_x)$, i.e. larger certified ellipses. (2) The vehicle example of Yin et al. (a $32$–$32$ tanh policy with saturation and LTI uncertainty):
| Multiplier | $\min_\delta\mathrm{trace}(P_x)$ | $\delta_{\max}$ | decision variables | solve time |
|---|---|---|---|---|
| diagonal circle (static) | 3.842 | 0.67 | 98 | 0.88 s |
| causal ZF, $\ell=1$ (= off-by-one IQC) | 2.726 | 1.47 | 2,754 | 123.9 s |
| acausal ZF, $\ell=1$ | 2.696 | 1.51 | 9,442 | 2,095 s |
4. Offset-Free Setpoint Tracking With NN Controllers (Pauli et al.)
Stabilising one equilibrium is rarely the job; a controller must bring the output to a setpoint $r$, and every $r$ has its own equilibrium. Section 2 would have to be redone for each of them, and Yin et al. fixed the biases to zero precisely to keep $x_*=0$. Pauli, Köhler, Berberich, Koch & Allgöwer, L4DC 2021 add the oldest trick in control, an integrator, and obtain certificates that cover whole sets of references.
In error coordinates $\tilde\chi=\chi-\chi_*$ the reference drops out of the linear part: $\tilde\chi_{t+1}=\tilde A\tilde\chi_t+\tilde B\big(\kappa(x_t,r)-\kappa(x_*,r)\big)$ and $y_t-r=\tilde C\tilde\chi_t$, since $B_rr$ is the same at every time. It still enters through the equilibrium $v_*(r)$ at which the activations are shifted. The activations are described by the slope QC around the equilibrium, $\alpha\le\frac{\varphi(v_{i})-\varphi(v_{*,i})}{v_i-v_{*,i}}\le\beta$ with $0\le\alpha\lt\beta$, stacked with a diagonal multiplier; the paper points out that the richer coupled multipliers once claimed for LipSDP are invalid, by the counterexample of Pauli et al., L-CSS 2022 (see Module 12).
Assumptions used: $A_a$ square and nonsingular (one steady state per $r$), $k_\xi$ invertible, $n_u=n_r$, and every activation slope-restricted on all of $\mathbb R$ with bounds $0\le\alpha\lt\beta$ (the paper uses one pair $[\alpha,\beta]$ as slope bounds and as the sector bounds they imply around every equilibrium). In words: one quadratic Lyapunov shape works around every equilibrium that a reference can induce, and the integrator turns convergence into zero offset.
Why one LMI covers all references. Slope restriction is incremental: the sector $[\alpha,\beta]$ holds around every point of the graph, so the QC around $v_*(r)$ has the same matrix for every $r$, and the LMI contains no $r$. The Lyapunov function $V_r(\chi)=(\chi-\chi_*(r))^{\top}P(\chi-\chi_*(r))$ changes with $r$, its matrix does not.
Networks trained on data from a bounded region are rarely globally stabilising (in the paper's example Theorem 1 is infeasible), so the paper localises the analysis in two ways.
A set of references. The steady-state map is linear, $x_*(r)=\Gamma r$ (from $A_a^{-1}$). Centre the box at $v^1_*(r_{\rm nom})$ for a nominal $r_{\rm nom}$ and compute the bounds there. In the coordinates $e=\chi-\chi_*(r)$, $s=r-r_{\rm nom}$ the first-layer deviation is affine, $v^1-v^1_*(r_{\rm nom})=Le+Js$ with $L=W_0H_x[I\ \ 0]=N^1_0$ and $J=W_0(H_x\Gamma+H_r)$ (derivation below). If the same LMI holds and, for some $Q\succ0$, $$\begin{bmatrix}d_j^2&L_j&J_j\\ L_j^{\top}&P&0\\ J_j^{\top}&0&Q\end{bmatrix}\succeq0\qquad\text{for every first-layer neuron } j,$$ then for every pair $(\chi_0,r)$ in the joint certified set $$\begin{aligned}\mathcal E_{P,Q}(r_{\rm nom})=\Big\{(\chi,r):\ &(\chi-\chi_*(r))^{\top}P(\chi-\chi_*(r))\\ &+(r-r_{\rm nom})^{\top}Q(r-r_{\rm nom})\le1\Big\}\end{aligned}$$ the loop with the constant reference $r$ stays in its slice $\{\chi:(\chi,r)\in\mathcal E_{P,Q}\}$, converges exponentially to $\chi_*(r)$, and $y_t\to r$ (the local bounds must hold around every $v_*(r)$ involved; second remark below).
Why the assumptions: the tight local bounds (a larger lower bound than the global $0$) are what make the first LMI feasible where Theorem 1 fails, but they are true only on the box; the containment LMI keeps the state (and, in the second version, the reference) where they are true, and the proof is the one-step induction of Section 2. $Q\succ0$ makes the admissible set of references bounded, and $Q$ trades the size of the admissible initial states against the distance of $r$ from $r_{\rm nom}$.
From $v^1=W_0(H_xx+H_rr)+b_0$ and $x_*(r)-x_*(r_{\rm nom})=\Gamma s$, $$v^1-v^1_*(r_{\rm nom})=W_0H_x\big(x-x_*(r)\big)+W_0H_x\Gamma s+W_0H_rs=Le+Js .$$ The Schur complement of the LMI is $L_jP^{-1}L_j^{\top}+J_jQ^{-1}J_j^{\top}\le d_j^2$, and Cauchy–Schwarz with the weight $\mathrm{blkdiag}(P,Q)$, as in Section 2, gives $|L_je+J_js|\le\big(L_jP^{-1}L_j^{\top}+J_jQ^{-1}J_j^{\top}\big)^{1/2}\big(e^{\top}Pe+s^{\top}Qs\big)^{1/2}\le d_j$ on $\mathcal E_{P,Q}$. At $e=0$ this also keeps every equilibrium input $v^1_*(r)$ of the set inside the box. For state feedback ($H_r=0$) the LMI is the paper's (9b), whose $M$ is our $\Gamma$. The set is an ellipsoid in $(e,s)$, but not in $(\chi,r)$: $x_*(r)$ is linear in $r$, while $\xi_*(r)=k_\xi^{-1}\big(u_*(r)-\kappa(x_*(r),r)\big)$ contains the network, so $(\chi,r)$-space holds a curved family of ellipsoidal slices. The LMIs stay convex because they are written in $(e,s)$; the governor problem below, a search over $r$, is in general not convex.
Two remarks make these statements usable. First, for output-error feedback the first-layer input at steady state is $W_0(r-Cx_*)+b_0=b_0$ for every $r$, so the bounds are the same for all references and the one-reference theorem already certifies every $r$ with an ellipsoid of fixed shape centred at $\chi_*(r)$: local stability depends only on the tracking error. Second (our remark; the paper computes the bounds for $r_{\rm nom}$ and leaves this implicit), for a set of references the sector bounds must hold around every equilibrium $v_*(r)$ in the set, not only around $v_*(r_{\rm nom})$; slope bounds over the box (smallest and largest derivative) have this property, because the joint containment LMI keeps $v_*(r)$ itself in the box.
The example. A linearised inverted pendulum ($m=0.15$ kg, $L=0.5$ m, friction $0.5$) discretised with $T_s=0.02$ s, controlled by a state-feedback network with two tanh layers of 5 neurons, trained by supervised learning to imitate an MPC with $|u|\le1$ on trajectories from $x_0\sim\mathcal U(-0.5,0.5)$ (initial states drawn uniformly, i.e. each coordinate independently from $[-0.5,0.5]$; Primer C), and $k_\xi=1$. The global theorem is infeasible. With $d_i=0.345$ the one-reference theorem ($r=0$, 1.14 s) and the set-of-references theorem ($r_{\rm nom}=0$, 0.50 s) are feasible; every reference angle $r\in[-0.2,0.2]$ can be tracked, the union of ellipsoids is much larger than the single one, and a reference governor tracks the closest admissible point to $r=-1$, while the same loop without the governor diverges.
5. Dissipativity Analysis and Training of RNNs (Pauli et al.)
A recurrent network is itself a dynamical system, and its robustness is an input–output property: how much can a perturbation of the input sequence change the output sequence? Pauli, Berberich & Allgöwer, at – Automatisierungstechnik 70(8):730–739, 2022 give a comprehensive introduction to the analysis of RNNs with robust control and dissipativity theory, use the $\mathcal H_2$ performance and the $\ell_2$ gain as robustness measures with respect to input perturbations, and present LMI constraints for training robust RNNs. For an LTI system with impulse-response matrices $G_t$ the squared $\mathcal H_2$ norm is the energy of the impulse response, $\|G\|_{\mathcal H_2}^2=\sum_t\|G_t\|_F^2$: an average over frequencies, whereas the $\ell_2$ gain is a worst case over inputs (Primer D). For a nonlinear RNN an $\mathcal H_2$-type measure needs its own definition; the construction below, the standard one behind such results, certifies only the $\ell_2$ gain.
Step 1. Multiply the LMI by $\zeta_t=(x_t,w_t,u_t)$ on both sides. The three terms become, in order, $V(x_{t+1})-V(x_t)$ (because $x_{t+1}=Ax_t+B_1w_t+B_2u_t$), $\|y_t\|^2-\gamma^2\|u_t\|^2$ (because $y_t=C_2x_t+D_{21}w_t+D_{22}u_t$), and $2w_t^{\top}T(v_t-w_t)=2\sum_i\lambda_iw_{t,i}(v_{t,i}-w_{t,i})$.
Step 2. For activations in the sector $[0,1]$ with $\varphi(0)=0$ (tanh, ReLU) each $w_i(v_i-w_i)\ge0$, so the third term is nonnegative and $V(x_{t+1})-V(x_t)\le\gamma^2\|u_t\|^2-\|y_t\|^2$: dissipativity with the $\ell_2$-gain supply.
Step 3. Sum from $0$ to $N-1$ with $x_0=0$, $V(0)=0$ (as for this quadratic storage), and use $V\ge0$: $\sum_{t\lt N}\|y_t\|^2\le\gamma^2\sum_{t\lt N}\|u_t\|^2$ for every $N$. If $D_{11}\ne0$ the layer is implicit, and this argument presupposes that it is well posed (that $w_t$ exists and is unique); the sector bound alone does not give that. The $(w,w)$ block contains $TD_{11}+D_{11}^{\top}T-2T$. If this is negative definite (which forces $T\succ0$) and the activations are slope-restricted in $[0,1]$, as tanh and ReLU are, the implicit layer is well posed (Section 1; REN paper, eq. 12 under its Assumption 1). A sector bound is not enough: $\varphi(v)=v(1+\sin10v)/2$ lies in the sector $[0,1]$, and with $D_{11}=0.5$, $T=1$ one has $2T-TD_{11}-D_{11}^{\top}T=1\gt0$, yet $v=0.5\,\varphi(v)+0.39$ has three solutions ($v\approx0.409,\ 0.736,\ 0.779$).
Incremental version and contraction. The sector QC compares one trajectory with the equilibrium $0$. Replacing it by the slope QC applied to the difference of two trajectories, $(\Delta v,\Delta w)$, with a diagonal $T$, turns the same LMI into a bound on the incremental gain, $\sum_t\|\Delta y_t\|^2\le\gamma^2\sum_t\|\Delta u_t\|^2+V(\Delta x_0)$: the RNN is $\gamma$-Lipschitz as a map between input and output sequences, up to a term from the initial states, and if the LMI holds strictly it also forgets its initial conditions (contraction). This is exactly the certificate built into the recurrent equilibrium network (Revay, Wang & Manchester, TAC 2024, Thm. 1): an incremental Lyapunov function $V(\Delta x)=\|\Delta x\|_P^2$ with $V(\Delta x_{t+1})\le\alpha^2V(\Delta x_t)-\Gamma(\Delta v_t,\Delta w_t)$ for contraction at some rate $\alpha\lt\bar\alpha$ (their eq. 17; $\Gamma\ge0$ is the incremental QC with a diagonal multiplier), and an incremental dissipation inequality for the incremental IQC defined by $(Q,S,R)$ with $Q\preceq0$ (their Lipschitz case is $Q=-\gamma^{-1}I$, $S=0$, $R=\gamma I$, a rescaling of ours).
Compare two trajectories with the same input, $\Delta u_t=0$. A strict LMI stays negative definite after adding $\varepsilon I$ for some $\varepsilon\gt0$; multiplying by $(\Delta x_t,\Delta w_t,0)$ and dropping the nonnegative terms $\|\Delta y_t\|^2$ and the slope QC gives $V(\Delta x_{t+1})\le V(\Delta x_t)-\varepsilon\|\Delta x_t\|^2\le\big(1-\varepsilon/\lambda_{\max}(P)\big)V(\Delta x_t)$, using $V(\Delta x)\le\lambda_{\max}(P)\|\Delta x\|^2$ (and $P\succ0$). Iterating and using $\lambda_{\min}(P)\|\Delta x\|^2\le V(\Delta x)$ gives $\|\Delta x_t\|^2\le\frac{\lambda_{\max}(P)}{\lambda_{\min}(P)}\big(1-\varepsilon/\lambda_{\max}(P)\big)^t\|\Delta x_0\|^2$: the difference of initial states is forgotten exponentially. In the REN theorem, $\bar\alpha\in(0,1]$ is a prescribed bound on the contraction rate, unrelated to the sector bound $\alpha$ of this page.
Training under the constraint. The weights $(A,B_1,C_1,D_{11},\dots)$ appear multiplied by $P$ and $T$, so the LMI is not jointly convex in weights and certificate (REN paper, Remark 4). Three remedies appear in this literature: fix the multipliers and enforce the LMI during training by ADMM (Pauli et al., L-CSS 2022) or by a barrier term in the loss (Pauli et al., CDC 2022, which reports that barrier training is much faster than ADMM); change variables so that the constraint is jointly convex (the convex parameterisations of the REN paper, Sec. IV); or parameterise the weights so that the LMI holds for every parameter value, which makes training unconstrained (REN direct parameterisation, Sec. V; Module 13).
6. Synthesis With Guarantees: Dissipativity-Constrained RL and Youla-REN
Analysis certifies a given network. Synthesis must search over networks while keeping the certificate, and the difficulty is algebraic: the closed-loop matrices are affine in the controller parameters, but the certificate multiplies them by $P$ and $T$, so the design condition is a bilinear matrix inequality (BMI). Two recent answers take opposite routes: convexify and project (Junnarkar, Arcak & Seiler), or build the certificate into the parameterisation (Youla-REN).
Projection-based RL under a closed-loop dissipativity LMI
Junnarkar, Arcak & Seiler, Automatica 193 (2026) 113185 synthesise neural controllers that maximise an RL reward subject to the hard constraint that the closed loop is dissipative. Their setting is continuous time: a dot means $d/dt$, and the one-step storage difference of the earlier sections becomes the derivative $\frac{d}{dt}(x^{\top}Px)=\dot x^{\top}Px+x^{\top}P\dot x=2x^{\top}P\dot x$ (product rule, Primer B). The plant is an LTI system $G_p$ in feedback with an uncertainty $\Delta_p$ (unmodelled dynamics and nonlinearities) described by a hard IQC, with disturbance $d$, performance output $e$, control $u$ and measurement $y$; the interconnection is assumed well posed (their eq. 1):
The controller is a recurrent implicit neural network (RINN); the subscript $k$ labels the controller, not a time step:
with activations sector-bounded and slope-restricted in $[0,1]$. With $A_k,B_{kw},B_{ky},C_{kv},C_{ku}$ all zero it is a static implicit network, and a feedforward network is the case of a strictly block lower triangular $D_{kvw}$ (their Example 1). Performance is dissipativity with a quadratic supply $s(d,e)=\begin{bmatrix}d\\ e\end{bmatrix}^{\top}X\begin{bmatrix}d\\ e\end{bmatrix}$, $X=\begin{bmatrix}X_{dd}&X_{de}\\ X_{de}^{\top}&X_{ee}\end{bmatrix}$, and a quadratic storage $x^{\top}Px$ (stability: $s=0$ with $P\succ0$; $\ell_2$ gain: $\gamma^2\|d\|^2-\|e\|^2$; passivity: $d^{\top}e$). The activations enter through the sector QC with multiplier $\begin{bmatrix}0 & \Lambda\\ \Lambda & -2\Lambda\end{bmatrix}$, $\Lambda\succeq0$ diagonal, and the plant uncertainty through a static IQC with multiplier $M_{\Delta_p}=\begin{bmatrix}M_{\Delta_p,vv}&M_{\Delta_p,vw}\\ M_{\Delta_p,vw}^{\top}&M_{\Delta_p,ww}\end{bmatrix}$ (dynamic IQCs of the class below are first converted to this form).
Statement. For fixed $M_{\Delta_p}$ and $X$, there exist a controller and a quadratic storage that satisfy their dissipation inequality (Lemma 2, the continuous-time analogue of the LMI of Section 5) with $P\succ0$ and $\Lambda\succ0$ if and only if there exist new variables $\hat\theta=\{S,R,N_A,N_B,N_C,$ $D_{kuw},\hat D_{kvy},\hat D_{kvw},\Lambda\}$ satisfying an LMI that is affine in $\hat\theta$ (their eq. 23, written out below), with $S\succ0$, $R\succ0$, $\Lambda\succ0$ and $\begin{bmatrix}R & I\\ I & S\end{bmatrix}\succ0$. The equivalence is between two certificates, not a description of all dissipative controllers.
The change of variables. Partition $P=\begin{bmatrix}S & U\\ U^{\top} & \star\end{bmatrix}$, $P^{-1}=\begin{bmatrix}R & V\\ V^{\top} & \star\end{bmatrix}$ ($\star$: blocks not needed by name), apply the congruence $\mathrm{diag}(Y,I)$ with $Y=\begin{bmatrix}R & I\\ V^{\top} & 0\end{bmatrix}$ and set $\hat D_{kvy}=\Lambda D_{kvy}$, $\hat D_{kvw}=\Lambda D_{kvw}$, $N_B=SB_{pu}D_{kuw}+UB_{kw}$, $N_C=\Lambda D_{kvy}C_{py}R+\Lambda C_{kv}V^{\top}$ and $$\begin{aligned}N_A&=\begin{bmatrix}N_{A11}&N_{A12}\\ N_{A21}&N_{A22}\end{bmatrix}=\begin{bmatrix}SA_pR & 0\\ 0 & 0\end{bmatrix}\\ &\quad+\begin{bmatrix}U & SB_{pu}\\ 0 & I\end{bmatrix}\begin{bmatrix}A_k & B_{ky}\\ C_{ku} & D_{kuy}\end{bmatrix}\begin{bmatrix}V^{\top} & 0\\ C_{py}R & I\end{bmatrix}.\end{aligned}$$ The controller is recovered from $\hat\theta$ by factorising $VU^{\top}=I-RS$ (for instance from an SVD, Primer A; invertible because $\begin{bmatrix}R & I\\ I & S\end{bmatrix}\succ0$) and solving these linear equations. This is the change of variables of Scherer, Gahinet and Chilali for $\mathcal H_\infty$ synthesis, extended to the multiplier $\Lambda$ of the network.
Why the assumptions: $X_{ee}\preceq0$ and $M_{\Delta_p,vv}\succeq0$ allow the factorisations $X_{ee}=-L_X^{\top}L_X$ and $M_{\Delta_p,vv}=L_{\Delta_p}^{\top}L_{\Delta_p}$, so a Schur complement removes the terms that are quadratic in the closed-loop matrices and leaves a condition that is bilinear only through $P$, $\Lambda$ and the controller; the plant-order controller makes the congruence $Y$ and the recovery of $(A_k,B_{kw},\dots)$ from $\hat\theta$ possible; the restriction on the IQC lets their Lemma 3 absorb the filters into an extended plant with a static IQC.
From the BMI to an LMI. Close the loop: state $x=(x_p,x_k)$, $w=(w_p,w_k)$, closed-loop matrices $\mathcal A,\mathcal B_w,\mathcal B_d,\mathcal C_v,\mathcal D_{vw},\dots$, affine in the controller. Lemma 2 asks that $2x^{\top}P\dot x+\begin{bmatrix}v\\ w\end{bmatrix}^{\top}M\begin{bmatrix}v\\ w\end{bmatrix}-s(d,e)\le0$ for all $(x,w,d)$, with $M$ collecting both QCs ($M_{vw}=\mathrm{diag}(M_{\Delta_p,vw},\Lambda)$, $M_{ww}=\mathrm{diag}(M_{\Delta_p,ww},-2\Lambda)$); integrating it, the hard IQC (zero filter state) makes the $M$-term's integral nonnegative, which leaves the dissipation inequality. Two terms are quadratic in the controller, $v_p^{\top}M_{\Delta_p,vv}v_p=\|L_{\Delta_p}v_p\|^2$ and $-e^{\top}X_{ee}e=\|L_Xe\|^2$; a Schur complement moves them into an off-diagonal block $G$ (their eq. 21). The substitution $x=Yz$ (the congruence) then turns every product of $P$ or $\Lambda$ with a controller matrix into one of the new variables (their eqs. 23–25). Order the columns as $(z,w_p,w_k,d)$, let $J_w$ pick $(w_p,w_k)$ and $J_d$ pick $d$. The blocks are
and, for $a=v,e$, the plant output $a$ as a function of $(z,w_p,w_k,d)$ (use $N_{A22}=D_{kuy}$ and $N_{A21}=C_{ku}V^{\top}+D_{kuy}C_{py}R$):
The multiplier cross terms, the second row being $\Lambda v_k$ in the new variables, are
With $\operatorname{He}(H)=H+H^{\top}$:
Every entry is affine in $\hat\theta$ (and in $M_{\Delta_p,ww}$, $X_{dd}$, which may therefore also be decision variables, e.g. to minimise $\gamma$). By the Schur complement the boxed LMI is $F+G^{\top}G\preceq0$, the congruence-transformed Lemma 2 inequality.
Recovering the controller. The upper-left block of $PP^{-1}=I$ reads $SR+UV^{\top}=I$, so $VU^{\top}=I-RS$. The LMI $\begin{bmatrix}R&I\\ I&S\end{bmatrix}\succ0$ gives $R-S^{-1}\succ0$ (Schur complement), so $S^{1/2}RS^{1/2}\succ I$ and $I-RS=S^{-1/2}\big(I-S^{1/2}RS^{1/2}\big)S^{1/2}$ is invertible. Factor it, e.g. with an SVD $I-RS=\Omega_1\Sigma\Omega_2^{\top}$, as $V=\Omega_1\Sigma^{1/2}$, $U=\Omega_2\Sigma^{1/2}$; both are invertible, hence so is $Y$, and $P=\begin{bmatrix}I&S\\ 0&U^{\top}\end{bmatrix}Y^{-1}$ (check: $PY=\begin{bmatrix}SR+UV^{\top}&S\\ U^{\top}R+\star V^{\top}&U^{\top}\end{bmatrix}$ and $U^{\top}R+\star V^{\top}=0$ is the lower-left block of $PP^{-1}=I$). Then $D_{kvy}=\Lambda^{-1}\hat D_{kvy}$, $D_{kvw}=\Lambda^{-1}\hat D_{kvw}$, $(A_k,B_{ky},C_{ku},D_{kuy})$ from $N_A$ by inverting its two invertible outer factors, $B_{kw}=U^{-1}(N_B-SB_{pu}D_{kuw})$ and $C_{kv}=\Lambda^{-1}(N_C-\hat D_{kvy}C_{py}R)V^{-\top}$. With the zero supply the nonstrict LMI gives a nonincreasing storage (stability), not convergence.
- Initialise the controller parameters $\theta$ arbitrarily (randomly, or warm-started), and $P,\Lambda\leftarrow I$ (or a known certificate of $\theta$). // $\theta$ need not be certified yet
- For $i=1,2,\dots$:
- $\theta'\leftarrow$ one reinforcement-learning step on $\theta$ // reward only; the training environment may differ from the design model
- If some $(P',\Lambda')$ certifies $\theta'$ (an LMI in $(P,\Lambda)$ for fixed $\theta'$): $\theta,P,\Lambda\leftarrow\theta',P',\Lambda'$.
- Else: build $\hat\theta'$ from $(\theta',P,\Lambda)$; project it onto the convex set of Theorem 4 (first the minimum Frobenius distance $\delta^*$, a projection as in Primer B; then a re-solve that maximises $\epsilon_{RS}$ in $\begin{bmatrix}R&I\\ I&S\end{bmatrix}\succ\epsilon_{RS}I$ subject to distance $\le\beta\delta^*$, $\beta\ge1$, which improves the conditioning of $I-RS$);
- extract $P,\Lambda$ from the projection and set $\theta\leftarrow\arg\min\|\theta-\theta'\|$ over controllers certified by this $(P,\Lambda)$ // again an SDP
Every iterate that leaves the loop is certified for the design model; the initial $\theta$ is not, and $P,\Lambda=I$ only serve to build the first $\hat\theta'$ (the projection then produces a valid $(P,\Lambda)$). Well-posedness of the implicit layer is added as $\Lambda D_{kvw}+D_{kvw}^{\top}\Lambda-2\Lambda\prec0$ (their Remark 7), which is $\hat D_{kvw}+\hat D_{kvw}^{\top}-2\Lambda\prec0$, affine in the new variables. The authors estimate the cost of the dissipativity check as $\mathcal O\big((n_p^2+n_\phi)^2(n_p+n_\phi)^2\big)$ and of each projection as $\mathcal O\big((n_p+n_\phi)^6\big)$ flops ($n_p$ plant states, $n_\phi$ neurons), which is why the check runs first and the projections only when it fails. Examples: an inverted pendulum and a flexible rod on a cart. Reward maximisation is the soft objective and dissipativity the hard constraint, a certificate-level analogue of the constrained RL of Module 8.
Stable by design: the Youla-REN
Wang, Barbara, Revay & Manchester, IEEE L-CSS 7:91–96, 2023 take the opposite route. For a partially observed linear plant $x_{t+1}=Ax_t+Bu_t+d^x_t$, $y_t=Cx_t+d^y_t$ (stabilisable and detectable, Primer D), pick gains with $A-BK$ and $A-LC$ stable and augment the observer-based controller (see Primer D) with a nonlinear Youla parameter $\mathcal Q$:
Why it works. Subtracting the observer equation from the plant, the observer error obeys $(x-\hat x)_{t+1}=(A-LC)(x_t-\hat x_t)+d^x_t-Ld^y_t$, and $\tilde y_t=C(x_t-\hat x_t)+d^y_t$; neither equation contains $u$. Hence the innovation $\tilde y=T_2d$ does not depend on what $\mathcal Q$ outputs, and $\mathcal Q$ sits outside every feedback loop: stability reduces to a cascade of stable systems (see Primer D), and no LMI has to be solved during learning. Stabilisability and detectability are exactly what guarantee such gains $K,L$.
Because the REN is directly parameterised (every parameter vector gives a contracting, Lipschitz $\mathcal Q$; Module 13), learning is unconstrained: exact gradients through a differentiable model, or random search with a zeroth-order oracle (cost values only, no gradients; Primer B) as in RL. The paper reports faster convergence to better controllers than competing policy classes, with stability preserved during the learning transients. The guarantee is for the nominal linear model; with model error the innovation depends on $u$ again and robustness must be argued separately (for instance by small gain with the Lipschitz bound of $\mathcal Q$). The survey of Manchester, Wang & Barbara, Annu. Rev. 2026 collects these model classes and their use as observers and policies. A faster recurrent class from the same group, R2DN (Barbara, Wang & Manchester, CDC 2026), replaces the REN's equilibrium layer by a 1-Lipschitz feedforward network in feedback with an LTI system and reports up to an order of magnitude faster training and inference.
| Route | What is guaranteed, when | Plant class | Computation |
|---|---|---|---|
| Analysis LMI (Sections 2–4) | stability / ROA for the trained network, after training | LTI, perturbations via IQCs | one SDP per certificate |
| Constrained training (ADMM, barrier; Module 12) | the LMI at the end of training (ADMM); at every iterate with projection or a log-det barrier | LTI + QCs | SDP-type steps or barrier gradients |
| Projection-based RL (Junnarkar et al.) | closed-loop dissipativity of every accepted iterate | LTI + IQC uncertainty, full-order controller | check LMI, convex projection when it fails |
| Youla-REN | contraction and Lipschitz closed loop for every parameter value | known partially observed LTI | none beyond gradient steps |
7. Reachability of NN Loops and Verified Lyapunov Controllers
Stability says where trajectories end; safety asks where they go on the way. Two very different certificate styles address this: SDP relaxations built from the same QCs as above, and complete verification by bound propagation and branch-and-bound. Only the interface of the latter is needed here (Module 15 develops the algorithms). A verifier receives a bounded input region and a statement that must hold at every point of it. Bound propagation pushes sound lower and upper bounds on every intermediate value through the network; when they are inconclusive, branch-and-bound splits the problem, typically by fixing an undecided ReLU $w=\max(z,0)$ to its active piece ($z\ge0$, $w=z$) or its inactive piece ($z\le0$, $w=0$), and repeats on the pieces. The outcome is a proof, a counterexample, or "unresolved" when the time budget runs out.
Reach-SDP
Hu, Fazlyab, Morari & Pappas, CDC 2020 over-approximate the forward reachable sets of a linear time-varying plant $x_{t+1}=A_tx_t+B_tu_t+c_t$ with a projected ReLU policy $u_t=\mathrm{Proj}_{\mathcal U_t}(\pi(x_t))$, $\mathcal U_t$ a box, so that the projection clips each input coordinate to its interval (Primer B). The exact one-step image of a set $\mathcal X$ is $\mathcal R_t(\mathcal X)=\{A_tx+B_t\mathrm{Proj}_{\mathcal U_t}(\pi(x))+c_t:\ x\in\mathcal X\}$, and the goal is a simple set $\bar{\mathcal R}$ that contains all of it. The projection is itself written as two extra ReLU layers, $x^{\ell+1}=\max(W_\ell x^\ell+b_\ell-\underline u,0)+\underline u$ and $x^{\ell+2}=-\max(\bar u-x^{\ell+1},0)+\bar u$, so the whole loop map is a ReLU network plus linear algebra. Three families of QCs describe it, all in the basis $\boldsymbol\xi=(x_0,\text{all neurons},1)$, whose constant entry lets a quadratic form carry affine and constant terms, $[x;1]^{\top}\begin{bmatrix}G&g\\ g^{\top}&g_0\end{bmatrix}[x;1]=x^{\top}Gx+2g^{\top}x+g_0$. The initial set: for a polytope $\{Hx\le h\}$ (Primer B) every product of two entries of $s=Hx-h\le0$ is nonnegative, so $s^{\top}\Gamma s\ge0$ for every symmetric $\Gamma$ with nonnegative entries (entrywise, not positive semidefinite). Each ReLU: $y_i\ge x_i$, $y_i\ge0$, $y_i^2=x_iy_i$, which is exact, because $y_i(y_i-x_i)=0$ leaves $y_i=0\ge x_i$ or $y_i=x_i\ge0$, i.e. $y_i=\max(x_i,0)$; and the coupled repeated-ReLU constraints $\lambda_{ij}\big[(y_j-y_i)(x_j-x_i)-(y_j-y_i)^2\big]\ge0$ with $\lambda_{ij}\ge0$. The candidate output set: $\{y:[y;1]^{\top}S[y;1]\le0\}$.
Take $E\succ0$ symmetric. The map $y\mapsto Ey+f$ sends the ellipsoid onto the unit ball, so its volume is the unit-ball volume divided by $\det E$; minimum volume means minimising $-\log\det E$, a convex function ($\log\det$ is concave, Primer B). Write the successor as $y=F\boldsymbol\xi$ and use $e_0^{\top}\boldsymbol\xi=1$ for the constant entry: then $Ey+f=\Omega\boldsymbol\xi$ with $\Omega=EF+fe_0^{\top}$, affine in $(E,f)$, and $\|Ey+f\|^2-1=\boldsymbol\xi^{\top}(\Omega^{\top}\Omega-e_0e_0^{\top})\boldsymbol\xi$, so $M_{\rm out}(S)=\Omega^{\top}\Omega-e_0e_0^{\top}$. This is quadratic in $(E,f)$, but by the Schur complement (with $-I\prec0$) $$\begin{gathered}M_{\rm in}(P)+M_{\rm mid}(Q)-e_0e_0^{\top}+\Omega^{\top}\Omega\preceq0\\ \iff\begin{bmatrix}M_{\rm in}(P)+M_{\rm mid}(Q)-e_0e_0^{\top}&\Omega^{\top}\\ \Omega&-I\end{bmatrix}\preceq0,\end{gathered}$$ which is affine in $(P,Q,E,f)$. Contrast Section 2: there the ellipsoid is an inner set in which $P$ enters linearly, and maximising its volume means minimising $\log\det P$, a concave function; here the outer set has a convex volume objective.
The paper verifies a double integrator (sampling time 1 s, $|u|\le1$) under a ReLU network with hidden layers of 10 and 5 neurons trained on 2420 samples of a linear MPC: every $x_0\in[2.5,3]\times[-0.25,0.25]$ reaches $[-0.25,0.25]^2$ within 6 steps without violating the state constraint, and the polytopic over-approximations are tight. A second example certifies a network that imitates a nonlinear MPC for a 6-D quadrotor.
Formally verified neural Lyapunov controllers
Yang, Dai, Shi, Hsieh, Tedrake & Zhang, ICML 2024 drop the linear plant and the quadratic Lyapunov function. For $x_{t+1}=f(x_t,u_t)$ with continuous $f$ they train a controller $u=\mathrm{clamp}(\phi_\pi(x)-\phi_\pi(x_*)+u_*,u_{\rm lo},u_{\rm up})$, for output feedback also a neural observer $\hat x_{t+1}=f(\hat x_t,u_t)+\phi_{\rm obs}(\hat x_t,y_t-h(\hat x_t))-\phi_{\rm obs}(\hat x_t,0)$, and a Lyapunov function $V$ that is positive definite by construction, on the internal state $\xi$ ($\xi=x$, or $\xi=(x,\hat x-x)$ for output feedback).
Why the disjunction. Earlier works demanded decrease on all of $\mathcal B$ and then took the largest sublevel set inside $\mathcal B$, which wastes the part of $\mathcal B$ outside $\mathcal S$ (not invariant anyway) and yields a smaller certified set. Here decrease is required only where $V\lt\rho$, and the condition is still stated on the explicitly defined box $\mathcal B$ that a verifier can handle.
The condition is checked by $\alpha,\beta$-CROWN (bound propagation plus branch-and-bound; Xu et al., ICLR 2021, Wang et al., NeurIPS 2021), extended to the nonlinear operations of the dynamics, and the largest verifiable $\rho$ is found by bisection. Training alternates adversarial search for counterexamples with a loss that also enlarges $\mathcal S$. The authors report larger verified regions than prior neural Lyapunov methods and, to their knowledge, the first formally verified neural output-feedback controllers with Lyapunov certificates. Context: Module 11.
8. Towards Scale: ReLU IQCs and Incremental Analysis (2025–2026)
Two recent preprints from the Seiler–Hu–Dullerud collaboration push the IQC programme in the two directions that matter most: tighter descriptions of the activation, and certificates whose size does not grow with depth. Both are arXiv preprints; statements may change between versions.
ReLU-specific hard IQCs. Vahedi Noori, Hu, Dullerud & Seiler (arXiv, Nov. 2025) analyse internal stability of a discrete-time feedback system with a ReLU nonlinearity, motivated by recurrent networks. After reviewing static QCs for slope-restricted nonlinearities they derive hard IQCs for the scalar ReLU from FIR filters and structured matrices, combine them with a dissipation inequality into an LMI, prove that the new IQCs form a superset of the Zames–Falb IQCs for slope-restricted nonlinearities, and report less conservative stability margins than Zames–Falb multipliers and static QCs, sometimes dramatically so.
Why the IQC holds. Stack the samples up to time $N$. The sum becomes a quadratic form in $a=w-v\ge0$ and $b=w\ge0$, namely $a^{\top}Q_1a+b^{\top}Q_2b+2b^{\top}Q_3a$ with banded Toeplitz matrices $Q_j$ built from the $m^j_i$. The first two terms add nonnegative coefficients times nonnegative products; in the third, every product $b_ta_s$ with $t\ne s$ has a coefficient $\ge0$, and the products with $t=s$ vanish because $w_t(w_t-v_t)=0$. Zames–Falb as a special case: $m^1=m^2=0$ and $m^3_i=-m_i$ reproduce every Zames–Falb multiplier of the same order ($m_i\le0$ for $i\ne0$, $\sum_im_i\ge0$), and the ReLU class does not even need the sum condition.
Depth-independent incremental certificates. Wang, Seiler, Dullerud & Hu (arXiv, Sept. 2026) consider an LTI plant $G$ in feedback with a deep feedforward network and an uncertainty $\Delta_U$ described by incremental $\rho$-hard IQCs. With nonzero biases the equilibrium may move or not exist, so they study pairs of trajectories (incremental convergence and incremental $\ell_2$ gain) instead of an equilibrium.
For layers $h^l=\phi(W_lh^{l-1}+b_l)$, $l=1,\dots,N$ (their indexing, $h^0=v$ the network input, the last layer's output $h^N=w$ taken directly as the network output; a linear readout is absorbed into the plant), activations slope-restricted in $[0,1]$ with $\phi(0)=0$, the whole network satisfies the $\Lambda$-incremental QC
where $\delta v=\delta h^0$ is the difference of the two network inputs (not the stack of pre-activations of Section 1) and $\delta h=(\delta h^1,\dots,\delta h^N)$, because the form equals $\sum_l2(\delta h^l)^{\top}\Lambda_l(W_l\delta h^{l-1}-\delta h^l)=\sum_{l,i}2\lambda_{l,i}\,\delta h^l_i(\delta z^l_i-\delta h^l_i)$, with $\delta z^l=W_l\delta h^{l-1}$ the pre-activation increment (the bias cancels), and slope $[0,1]$ gives $\delta h_i(\delta z_i-\delta h_i)\ge0$ neuron by neuron. The full-order SDP built from it (their Thm. 1) grows with the total number of neurons. The paper decomposes it and combines the decomposition with scalable Lipschitz estimation for the inner layers, so that the remaining control-analysis LMIs depend only on the widths of the last two layers and not on the depth (computing the inner-layer certificates still visits every layer); the incremental small-gain test is recovered as a special case, and a multi-round alternating update of the coupling variables reduces conservatism. They also show that incremental convergence implies convergence of all trajectories to a common limit point under an extra assumption (existence of one convergent trajectory), with a counterexample without it.
If $\|x_t-y_t\|\to0$ for any two trajectories and one trajectory satisfies $y_t\to y_*$, then $\|x_t-y_*\|\le\|x_t-y_t\|+\|y_t-y_*\|\to0$ (triangle inequality): every trajectory has the same limit. Without a convergent reference, differences can vanish while every trajectory drifts, as the sequences $x^{(a)}_t=t+a\,2^{-t}$ show: $x^{(a)}_t-x^{(b)}_t=(a-b)2^{-t}\to0$, yet each tends to $\infty$. The paper's counterexample (Thm. 4) is a time-varying interconnection that satisfies the LMI although its trajectories converge to no fixed point.
Setting. The loop is well posed; $\Delta_U$ is causal, $\Delta_U(0)=0$, and satisfies incremental $\rho$-hard IQCs with filters started at zero; the $N\ge2$ layers have activations slope-restricted in $[0,1]$ with $\phi(0)=0$; the network input is a linear function of the plant-and-filter state $\zeta$, of $q$ and of $d$ (no direct dependence on $w$), and this map is absorbed into the first weight, giving $\widetilde W_1$. Stack $z=(\delta\zeta,\delta q,\delta d)\in\mathbb R^m$ and $w=\delta h^N\in\mathbb R^{n_N}$. For a storage $V=\delta\zeta^{\top}P\delta\zeta$ with $P\succ0$, IQC weights $\tau\ge0$ (their $\alpha$) and $\gamma\gt0$, collect the plant part of the dissipation inequality in the matrix $\begin{bmatrix}\mathscr A&\mathscr B\\ \mathscr B^{\top}&\mathscr C\end{bmatrix}$ whose quadratic form in $(z,w)$ is $V(\delta\zeta^+)-V(\delta\zeta)+\delta r^{\top}M(\tau)\delta r+\|\delta e\|^2-\gamma\|\delta d\|^2$; it is affine in $(P,\tau,\gamma)$.
Statement. Let diagonal $\Lambda_1,\dots,\Lambda_{N-1}\succeq0$ and a symmetric $Y_1$ make the network QC matrix of the first $N-1$ layers (built with $\widetilde W_1$) plus $\mathrm{diag}(Y_1,0)$ negative definite, a Lipschitz-type certificate that scalable estimators (ECLipsE and its variants) compute layer by layer. Define
(every $\Xi_l\prec0$ by successive Schur complements, and $D\prec0$ because its first term is $Y_1^{-1}$). If some $P\succ0$, $\tau\ge0$, $\gamma\gt0$, diagonal $\Lambda_N\succeq0$, symmetric $Y_2$ and $Y_3\in\mathbb R^{m\times n_N}$ satisfy
then the full-order certificate holds, so for $d=0$ any two state trajectories converge to each other, and $\sum_{k\le K}\|\delta e_k\|^2\le\gamma\sum_{k\le K}\|\delta d_k\|^2+V(\delta\zeta_0)$ for every $K$: incremental $\ell_2$ gain at most $\sqrt\gamma$ (here, as in the paper, $\gamma$ bounds the squared gain, unlike the notation box).
Why it works. The full LMI is a plant part plus a network part. Subtracting a coupling matrix $Y=\begin{bmatrix}Y_1&Y_3\\ Y_3^{\top}&Y_2\end{bmatrix}$ from the first and adding it to the second changes nothing when the two are added back, and the full LMI holds if and only if both split LMIs hold for some $Y$ (their Lemma 1). Eliminating the inner network blocks by successive Schur complements (the $\Xi_l,\Gamma_l$ recursion) turns the network part into the second LMI above. The two LMIs have sizes $m+n_N$ and $m+n_{N-1}+n_N$, whatever the depth.
Fix $\gamma$ and $Y_1\prec0$, compute $\Lambda_1,\dots,\Lambda_{N-1}$ with a scalable estimator, and minimise $s$ over $(P,\tau,\Lambda_N,Y_2,Y_3)$ subject to the first reduced LMI relaxed to $\preceq sI$ and the second kept strict. If $s\lt0$ (with a numerical margin), the certificate is found. Otherwise fix all $\Lambda_l$, $Y_2$, $Y_3$ and eliminate the network blocks from the output layer backwards ($2\le l\le N$):
With all $\Xi'_l\prec0$, the network LMI is equivalent to $Y_1\prec\Theta$, so one minimises $s$ over $(P,\tau,Y_1)$ subject to $Y_1\prec\Theta$ and the plant LMI $\preceq sI$. The old $Y_1$ is feasible, so this step cannot increase $s$. Then the upstream certificate is recomputed for the new $Y_1$ (which must stay $\prec0$ for the estimators) and the rounds repeat up to a limit. Recomputing the upstream certificate carries no such monotonicity guarantee, and running out of rounds returns "unknown", not "infeasible".
- Richer incremental multipliers. The repeated-nonlinearity (coupled) multipliers fail incrementally; which larger classes are valid for Lipschitz and contraction certificates, and how tight can they be?
- Synthesis without BMIs beyond full-order changes of variables and direct parameterisations: reduced-order and structured controllers, output feedback with uncertain models, dynamic multipliers optimised jointly with the network.
- Local certificates with constraints: saturation, state constraints and safe sets (Module 10) together with regions of attraction, for moving references and in the presence of disturbances.
- Scale and architecture: convolutional and state-space layers (the 2-D systems view of Module 12), deep networks without layer-wise blow-up, and nonlinear plants beyond sector-bounded IQC models.
- Learned plants: combining statistical model-error bounds (Modules 3 and 11) with IQC certificates for the loop.
Walkthrough: Stability LMI for a Linear Plant With a One-Layer NN
The smallest loop in which every step of Section 2 is visible has one state and one neuron. The six steps derive the stability LMI, solve it in closed form, and turn the insight box of Section 1 into an exact statement: with a global sector, what is certified is exactly what the two extreme members of the sector allow, and only the local sector can certify an open-loop unstable plant.
Interactive: A Neural Controller in Closed Loop
The plant is a linearised pendulum, discretised by Euler's method with $h=0.1$: $x_{t+1}=\begin{bmatrix}1&h\\ ah&1-0.5h\end{bmatrix}x_t+\begin{bmatrix}0\\ bh\end{bmatrix}u_t$ (angle and rate; unstable for $a\gt0$). The controller is a bias-free tanh network with two neurons, $u=W_1\tanh(W_0x)$ with $W_0=w_0\begin{bmatrix}1&0.5\\ 0&1\end{bmatrix}$ and $W_1=w_1\begin{bmatrix}-6&-1.5\end{bmatrix}$: a saturating PD law (see Primer D) with linearisation $u\approx W_1W_0x$ and $|u|\le7.5\,w_1$. For every setting the page solves the SDP of Section 2 in the browser: a log-barrier interior-point method for the $4\times4$ LMI in $(P,\lambda_1,\lambda_2)$ and the two $3\times3$ containment LMIs, with a margin $\mathcal M\preceq-5\cdot10^{-5}\,\mathrm{trace}(P)\,I$. Feasibility is decided by a phase-I problem: with $\mathrm{trace}(P)=1$, minimise $s$ subject to $\mathcal M\preceq sI$ and $P\succeq-sI$. The stability LMI is homogeneous in $(P,T)$, so every certificate can be scaled to $\mathrm{trace}(P)=1$, and $s\lt0$ gives $P\succeq-sI\succ0$ and $\mathcal M\preceq sI\prec0$: $s\lt0$ is exactly a certificate, which is afterwards scaled up (shrinking the ellipse) until the containment LMIs hold. The page accepts only $s\lt-2\cdot10^{-4}$. Missing that threshold is not treated as a proof that no certificate exists: the panel shows the signed value of $s$ and claims impossibility only when a linear law inside the sector is unstable beyond rounding error (a spectral radius within $10^{-9}$ of $1$ is reported as too close to decide). In local mode it bisects for the largest box $|v_i|\le\delta$ on which the sector $[\tanh(\delta)/\delta,1]$ still admits a certificate and then minimises $\mathrm{trace}(P)$ over $\delta$ by golden-section search, as Pauli et al. do. The returned $(P,T)$ is re-checked with a Jacobi eigenvalue computation (plane rotations that drive the off-diagonal entries of a symmetric matrix to zero, so that the diagonal holds its eigenvalues up to rounding). Independently of all that, the true nonlinear loop is simulated, on a grid and from points on the certified ellipse.
The unknowns are $(p_{11},p_{12},p_{22},\lambda_1,\lambda_2)$, with $P=\begin{bmatrix}p_{11}&p_{12}\\ p_{12}&p_{22}\end{bmatrix}$ and $T=\mathrm{diag}(\lambda_1,\lambda_2)$. With $G=[A\ \ BW_1]$ (so $x^+=G(x,w)$) and $F=\mathrm{diag}(W_0,I_2)$ (so $(v,w)=F(x,w)$), the stability matrix is $\mathcal M=G^{\top}PG-\mathrm{diag}(P,0)+F^{\top}Q_{\alpha\beta}(T)F$ with $\beta=1$, a $4\times4$ matrix affine in the unknowns. No separate constraint $\lambda_i\ge0$ is needed: the $(w_i,w_i)$ diagonal entry of $\mathcal M$ is $(BW_1)_i^{\top}P(BW_1)_i-2\lambda_i$ ($(BW_1)_i$ the $i$-th column), and it is negative when $\mathcal M\prec0$, so $\lambda_i\gt\frac12(BW_1)_i^{\top}P(BW_1)_i\ge0$ for every accepted point.
Assume the linearised loop $A_c=A+BW_1W_0$ is Schur stable. On the box the sector is $[\alpha,1]$ with $\alpha=\tanh(\delta)/\delta\to1$ as $\delta\downarrow0$. Write each $w_i=mv_i+dq_i$ with $m=(1+\alpha)/2$ and $d=(1-\alpha)/2$; the sector says exactly $|q_i|\le|v_i|$, and with $T=\tau I$ the sector QC equals $2\tau d^2(\|v\|^2-\|q\|^2)$ because $w-\alpha v=d(v+q)$ and $v-w=d(v-q)$. Choose $P\succ0$ with $A_c^{\top}PA_c-P=-I$. At $d=0$ the loop is $x^+=A_cx$, and the stability form in $(x,q)$ plus $\lambda(\|W_0x\|^2-\|q\|^2)$ has the blocks $-I+\lambda W_0^{\top}W_0$ and $-\lambda I$, negative definite for $0\lt\lambda\lt1/\|W_0\|^2$. The matrix depends continuously on $d$, so it stays negative definite for all small $d\gt0$, i.e. for all small boxes; the change of variables $w=mW_0x+dq$ turns it into the LMI of Section 2 with $T=\frac{\lambda}{2d^2}I$. Scaling $(P,T)$ up then shrinks the ellipse into the box. The browser can still miss such a certificate: its smallest box is $\delta=0.02$ and it demands a margin.
From the mathematics to a real decision
Learning objectives
- Combine plant and neural-controller dynamics before making a stability claim.
- Verify state containment, actuator limits, and disturbance effects on the same operating region.
- Identify when sensor bias or controller memory changes the object that must be certified.
A commissioning decision
The enclosure's temperature reference is fixed at 40 degrees Celsius. Let $e_t=(T_t-40)/10$ be normalized temperature error, with one-minute sampling. A centered heater command $u_t$ is measured in kilowatts; negative command means reducing power relative to the equilibrium feedforward setting. Assume the plant and controller satisfy
The centered actuator permits $|u_t|\le0.8\,\mathrm{kW}$. The desired operating region is $|e_t|\le0.5$, corresponding to 35–45 degrees. Disturbance contributes at most $|w_t|\le0.02$ per update, or 0.2 degrees Celsius. Assume the plant equation is exact inside this region, the equilibrium feedforward really cancels the fixed ambient load, the controller has no delay, and the sensor initially reports $e_t$ exactly.
This is a deliberately simple neural nonlinearity. It isolates the feedback calculation without attributing a deployment guarantee to a trained network. The actuator requirement also makes the certificate local in its physical applicability even though the mathematical tanh expression is defined globally.
Worked decision, with its limits
Close the loop. The disturbance-free map is $F(e)=0.9e-0.3\tanh(e)$. Its derivative is $F'(e)=0.9-0.3\operatorname{sech}^2(e)$, which lies between 0.6 and 0.9. Since $F(0)=0$, the mean-value theorem gives $|F(e)|\le0.9|e|$. Thus a Euclidean storage $V(e)=e^2$ contracts by a factor at most $0.9^2=0.81$ in the nominal loop.
Add the disturbance before claiming containment. The physical loop obeys $|e_{t+1}|\le0.9|e_t|+0.02$. At the proposed boundary 0.5, the right side is 0.47, inside the region. Induction establishes sampled state containment from every allowed initial error. The limiting bound is $0.02/(1-0.9)=0.2$, or 2 degrees, rather than convergence to the reference under every persistent disturbance.
Check the actuator on that same region. For $|e|\le0.5$, the command magnitude is at most $1.5\tanh(0.5)\approx0.693176\,\mathrm{kW}$, below 0.8. Thus saturation does not change the assumed closed-loop equation on the invariant region. A global claim based on the unsaturated tanh controller would require more care, because its limiting command magnitude is 1.5 kilowatts.
Inspect a physical starting point. At 44 degrees, $e_0=0.4$. The nominal next error is $0.36-0.3\tanh(0.4)\approx0.246015$, giving temperature $42.460153$ degrees. Its energy changes from 0.16 to approximately 0.060524. With the worst positive disturbance, the next error is 0.266015, still covered. The uniform 0.47 boundary calculation is stronger evidence for the whole region than this one appealing trajectory.
The commissioning conclusion is sampled containment and an ultimate disturbance bound for the stated fixed reference. Continuous-time temperatures, setpoint changes, sensor errors, and controller timing are not silently included. Each can be incorporated, but the equations and resulting certificate must then be revised.
A tempting wrong approach
The controller alone has sensitivity at most 1.5 kilowatts per normalized error unit. Calling that gain safe or unsafe without the plant ignores both the negative feedback sign and the plant's 0.2 coefficient. The closed map has derivative at most 0.9. Conversely, an actuator saturation or delay can change that map even if the same network weights are deployed.
Transfer the argument
Exercise 14.B1 — Medium: Carry a bounded sensor bias through feedback
The sensor reports $e+b$ with $|b|\le0.03$, corresponding to 0.3 degrees. Keep the disturbance bound 0.02. Derive a sufficient ultimate error bound, test invariance of $|e|\le0.5$, and check the actuator limit.
Review: Containment of a certified region.
Show hint
Tanh is 1-Lipschitz. Treat the change from $\tanh(e)$ to $\tanh(e+b)$ as an additional bounded input to the closed map.
Show worked solution
The extra next-error magnitude is at most $0.3|b|\le0.009$. Therefore $|e_{t+1}|\le0.9|e_t|+0.029$, with ultimate bound 0.29, or 2.9 degrees. At the boundary the bound is $0.45+0.029=0.479\le0.5$, preserving invariance. The command magnitude is at most $1.5\tanh(0.53)\approx0.728072$ kilowatts, still below 0.8. Constant bias need not permit zero tracking error; the correct conclusion is the revised bound.
Exercise 14.B2 — Hard: A stable delayed loop can leave the temperature box
Consider instead the linear delayed controller $u_t=-1.5e_{t-1}$ with zero disturbance. Use augmented state $(e_t,e_{t-1})$ and matrix $A_d=\left[\begin{smallmatrix}0.9&-0.3\\1&0\end{smallmatrix}\right]$. Test all initial augmented states in the square, allowing controller memory to be initialized independently of the current temperature. Show that this linear loop is asymptotically stable but the square $|e_t|,|e_{t-1}|\le0.5$ is not invariant. No prior actuator-bounded history is required in this initialization model.
Review: The full closed-loop state.
Show hint
Compute the roots of $\lambda^2-0.9\lambda+0.3=0$. Then try histories with current and previous error of opposite signs.
Show worked solution
The roots are $0.45\pm0.312250i$, each with magnitude $\sqrt{0.3}\approx0.547723\lt1$. Hence all augmented trajectories converge to zero. Yet history $(0.5,-0.5)$ gives next error $0.9(0.5)-0.3(-0.5)=0.6$, outside the temperature limit. The command is 0.75 kilowatts and respects the 0.8 limit, so saturation does not explain the violation. Stability in the augmented state does not imply containment in the requested square. A valid delay-aware design needs an invariant region in both state coordinates whose projection satisfies the physical temperature limit.
Synthesis and bridge
A feedback certificate ties together an equilibrium, a plant, a controller, a state description, and an operating domain. The same weights can support a different conclusion after sensor bias or timing is changed. Storage decrease and physical containment should be read together, with the disturbance terms kept visible.
The final verification chapter returns to the neural map and asks whether a specified output inequality holds over every allowed input. Such a verifier can supply the missing local bounds used in a feedback argument, but the distinction between a function specification and an entire trajectory remains essential.
Exercises
Find an equilibrium, calculate a scalar Lyapunov decrease, and check an ellipsoid against a pre-activation box. Explain why local sector validity and invariance must be proved together.
If this is difficult, revisit the earlier prerequisite, work the Easy questions, and return to the linked section. You can postpone the advanced extensions while building confidence with the core certificate.
Graded practice: build the calculation, then audit the claim
These twelve new questions each include an independent hint and a fully worked solution. The difficulty measures the amount of reasoning, not the amount of notation. The original research exercises follow below.
Easy: read definitions and compute
Easy 1 — Find the equilibrium before shifting
For $x_{t+1}=0.5x_t+u_t$ and $u_t=0.25x_t+1$, find $(x_*,u_*)$ and the deviation dynamics. Is the origin an equilibrium of the unshifted system?
Review if needed: Primer D: discrete-time state equations; this module's relevant section.
Hint
Substitute the controller into the plant, then solve $x_*=x_{t+1}=x_t$.
Worked solution
Step 1. The closed loop is $x_{t+1}=0.75x_t+1$. At equilibrium $0.25x_*=1$, so $x_*=4$ and $u_*=0.25(4)+1=2$.
Step 2. Set $\tilde x_t=x_t-4$. Subtracting $4=0.75(4)+1$ gives $\tilde x_{t+1}=0.75\tilde x_t$.
Step 3. The origin is not an equilibrium: it moves to $1$ in one step. Stability must be studied around the actual equilibrium; subtracting the correct equilibrium removes the affine constants.
Easy 2 — A Lyapunov difference is a difference of energies
For $x^+=0.8x$ and $V(x)=2x^2$, compute $V(x^+)-V(x)$. Evaluate $V$ before and after one step from $x=2$.
Review if needed: Primer D: Lyapunov stability and invariant sets; this module's relevant section.
Hint
Square the factor $0.8$ when substituting into $V$.
Worked solution
Step 1. $V(x^+)=2(0.8x)^2=1.28x^2$. Thus $\Delta V=(1.28-2)x^2=-0.72x^2$, strictly negative for nonzero $x$.
Step 2. At $x=2$, $V=8$. The next state is $1.6$ and $V(x^+)=2(1.6)^2=5.12$, a decrease of $2.88$.
Step 3. The nonnegative energy decreases and the exact trajectory is $x_t=0.8^tx_0\to0$. The negative coefficient in the Lyapunov difference certifies convergence in this scalar example.
Easy 3 — An ellipsoid inside a pre-activation interval
For $P=\operatorname{diag}(4,1)$, describe the semi-axes of $x^\top Px\le1$. If a pre-activation is $v=2x_1+x_2$, find its maximum absolute value on this ellipsoid. Is the box $|v|\le1$ large enough?
Review if needed: Primer A: quadratic forms and PSD order; this module's relevant section.
Hint
The sharp squared bound is $aP^{-1}a^\top$ for the row $a=(2,1)$.
Worked solution
Step 1. The inequality is $4x_1^2+x_2^2\le1$, with semi-axes $1/2$ and $1$. The inverse is $P^{-1}=\operatorname{diag}(1/4,1)$.
Step 2. $aP^{-1}a^\top=4(1/4)+1=2$, so $|v|\le\sqrt2$. Equality occurs at $x=P^{-1}a^\top/\sqrt2=(1/(2\sqrt2),1/\sqrt2)^\top$.
Step 3. The interval $[-1,1]$ misses that attainable value. A local sector stated only there cannot be applied on the whole ellipsoid; the required half-width is at least $\sqrt2$.
Easy 4 — Two shifted ReLUs need not be repeated
Define $\tilde\varphi_1(s)=\operatorname{ReLU}(s+1)-1$ and $\tilde\varphi_2(s)=\operatorname{ReLU}(s-1)$. Check both values at $s=0$ and $s=1/2$. Are these the same scalar function?
Review if needed: Module 12: valid and invalid activation multipliers; this module's relevant section.
Hint
Both shifts remove the value at the basepoint, but the basepoints lie on different sides of the kink.
Worked solution
Step 1. At $s=0$, the values are $1-1=0$ and $\operatorname{ReLU}(-1)=0$. Both shifted maps vanish at the origin.
Step 2. At $s=1/2$, the first value is $1.5-1=0.5$, whereas the second is $\operatorname{ReLU}(-0.5)=0$. Hence the maps differ.
Step 3. Each remains slope-restricted in $[0,1]$. Individual sector/slope QCs survive the shift, but an argument that assumes one identical repeated function across these two channels does not.
Medium: connect two or three steps
Medium 1 — A sector bound and a slope bound differ
For tanh on $[-1,1]$, compute the centred lower sector bound $\alpha=\tanh(1)$ and lower slope bound $\mu=1-\tanh^2(1)$. Evaluate the $[0,1]$ sector form at $v=1,w=\tanh(1)$. Why does substituting $\mu$ for $\alpha$ lose information?
Review if needed: Primer B: derivatives and directional slopes; this module's relevant section.
Hint
A sector compares a point with the fixed equilibrium; a slope bound compares any two points.
Worked solution
Step 1. $\alpha\approx0.76159416$ is the smallest chord slope from the origin on this interval. The smallest derivative is $\mu\approx0.41997434$, attained at the endpoints.
Step 2. The global $[0,1]$ QC is $2w(v-w)$. Substitution gives $2\tanh(1)(1-\tanh(1))\approx0.363137\ge0$.
Step 3. The slope bound implies the weaker sector $[\mu,1]$, but its smaller lower endpoint admits more nonlinearities. The larger centred chord bound $\alpha$ is legitimate for the fixed equilibrium and can improve the local stability certificate.
Medium 2 — Propagate an affine interval with signed weights
A hidden-output box has centre $c=(1,-1)$ and radius vector $r=(0.2,0.5)$. The next pre-activation is $v=2w_1-3w_2+1$. Find its exact interval over this box and the corner attaining the upper endpoint.
Review if needed: Primer 0: maxima and bounds; this module's relevant section.
Hint
The centre is $ac+b$ and the radius is $|a|r$.
Worked solution
Step 1. The centre is $2(1)-3(-1)+1=6$. The radius is $2(0.2)+3(0.5)=1.9$. Thus $v\in[4.1,7.9]$.
Step 2. To maximize $v$, take $w_1=1.2$ because its coefficient is positive and $w_2=-1.5$ because its coefficient is negative. This gives $2.4+4.5+1=7.9$.
Step 3. This endpoint is exact over the independent-coordinate box. If the true hidden outputs are correlated, that corner may be unreachable; the interval remains sound but may be conservative.
Medium 3 — Why an unstable plant needs a local lower sector
Consider $x^+=1.2x-0.5w$ and the uncertainty family $w=dx$ for $d\in[\alpha,1]$, $0\le\alpha\le1$. Find when every member is asymptotically stable. Compare $\alpha=0$ and $\alpha=0.5$.
Review if needed: Primer D: discrete-time state equations; this module's relevant section.
Hint
The scalar multiplier $1.2-0.5d$ must lie strictly between $-1$ and $1$.
Worked solution
Step 1. The multiplier decreases with $d$ and ranges from $0.7$ to $1.2-0.5\alpha$. Its smallest value is already positive and below one.
Step 2. Uniform strict stability therefore requires $1.2-0.5\alpha\lt1$, or $\alpha\gt0.4$. At $\alpha=0.4$ the endpoint has multiplier one, so asymptotic stability fails.
Step 3. For $\alpha=0$ the family includes $d=0$, the unstable open loop with multiplier $1.2$. For $\alpha=0.5$, all multipliers lie in $[0.7,0.95]$. A valid local sector can exclude a controller that is effectively switched off.
Medium 4 — A hard IQC can have a negative time sample
Let $w_t=v_t/2$, $a_t=v_t-w_t$, $b_t=w_t$, with $a_{-1}=0$ and $(v_0,v_1)=(2,1)$. Evaluate $b_t(a_t-a_{t-1})$ at both times and its partial sums. Show why nonnegative sums are possible without nonnegative individual terms.
Review if needed: Primer 0: sequences and geometric sums; this module's relevant section.
Hint
Here $a_t=b_t=v_t/2$. Use $2a_t(a_t-a_{t-1})=a_t^2-a_{t-1}^2+(a_t-a_{t-1})^2$.
Worked solution
Step 1. $a_0=b_0=1$ and $a_1=b_1=1/2$. The terms are $1(1-0)=1$ and $(1/2)(1/2-1)=-1/4$. Their partial sums are $1$ and $3/4$.
Step 2. For any finite sequence, summing the identity in the hint gives $\sum_{t=0}^N a_t(a_t-a_{t-1})=\tfrac12a_N^2+\tfrac12\sum_{t=0}^N(a_t-a_{t-1})^2\ge0$. The initial square vanishes because $a_{-1}=0$.
Step 3. This is a finite-horizon, or hard, IQC for this relation. A Lyapunov proof may use its summed nonnegativity; deleting a negative sample as though the IQC held pointwise would change the argument.
Hard: combine calculations with assumptions
Hard 1 — Close the local-sector invariance argument
A scalar map is $x^+=x/2$ for $|x|\le1$ and $x^+=2x$ for $|x|\gt1$. With $V=x^2$ prove that $\{V\le1\}$ is invariant and contained in the ROA of zero. Does the same local calculation prove convergence from $x_0=1.1$?
Review if needed: Primer 0: inequalities and proof steps; this module's relevant section.
Hint
Use the local rule at the current state, then prove the next state remains in its domain.
Worked solution
Step 1. If $V(x_t)\le1$, then $|x_t|\le1$, so the local rule applies and $V(x_{t+1})=V(x_t)/4\le1/4\le1$. This proves invariance by induction from any initial state in the set.
Step 2. Induction also gives $V(x_t)=4^{-t}V(x_0)$, hence $x_t\to0$. Thus the entire sublevel set is in the ROA, using only the rule valid there.
Step 3. At $x_0=1.1$ the exterior rule applies and $x_t=1.1\,2^t$, which diverges. A certificate based on local constraints needs the containment/invariance step; its conclusion extends only over the certified initial-state set.
Hard 2 — Check a stability LMI entirely by hand
For $x^+=1.2x-0.5w$ and the sector $w/x\in[0.5,1]$, use $V=x^2$ and multiplier $T=1$. Expand $\Delta V+2(w-0.5x)(x-w)$ as a quadratic form. Prove its matrix is negative definite and explain the stability conclusion when the sector is local.
Review if needed: Module 2: the S-procedure; this module's relevant section.
Hint
For a symmetric $2\times2$ matrix, a negative first diagonal entry and positive determinant imply negative definiteness.
Worked solution
Step 1. The Lyapunov difference is $0.44x^2-1.2xw+0.25w^2$. The QC is $-x^2+3xw-2w^2$. Their sum is $-0.56x^2+1.8xw-1.75w^2$.
Step 2. The matrix is $M=\begin{bmatrix}-0.56&0.9\\0.9&-1.75\end{bmatrix}$. Its determinant is $0.98-0.81=0.17\gt0$ and its first diagonal is negative, so $M\prec0$ (eigenvalues approximately $-2.23390,-0.0761001$).
Step 3. On the sector, the QC is nonnegative, so $\Delta V\le\Delta V+\mathrm{QC}\lt0$ away from zero. For a local sector, one must also exhibit a sublevel set contained in its pre-activation domain and use invariance; the LMI alone does not supply a global conclusion.
Hard 3 — A disturbance changes convergence into a bound
Suppose $V(x)=\|x\|_2^2$ and a closed-loop certificate proves $V_{t+1}\le0.5V_t+2d_t^2$ with $|d_t|\le0.1$. Starting with $V_0\le0.25$, derive a finite-time bound, prove invariance of $V\le0.25$, and state the limiting state-radius bound.
Review if needed: Primer D: Lyapunov stability and invariant sets; this module's relevant section.
Hint
Unroll the recursion and sum a geometric series. A persistent disturbance need not vanish.
Worked solution
Step 1. The disturbance term is at most $0.02$. Recursion gives $V_t\le0.5^tV_0+0.02\sum_{j=0}^{t-1}0.5^j=0.5^tV_0+0.04(1-0.5^t)$.
Step 2. If $V_t\le0.25$, then $V_{t+1}\le0.125+0.02=0.145\le0.25$. Hence the sublevel set is invariant for every allowed disturbance sequence.
Step 3. Taking a limit superior gives $\limsup_tV_t\le0.04$, so $\limsup_t\|x_t\|_2\le0.2$. This proves a disturbance-dependent ultimate bound, not convergence to zero under arbitrary persistent disturbance.
Hard 4 — Audit three tempting certificate shortcuts
A reviewer proposes: (i) couple all shifted ReLU channels with equilibrium pre-activations $(0,0,1)$ as one repeated nonlinearity; (ii) treat every time sample of a hard IQC as nonnegative; (iii) reuse a fixed-network stability certificate after a small unbounded-in-the-proof weight update. For each proposal identify the missing justification and a valid repair.
Review if needed: Module 12: valid and invalid activation multipliers; this module's relevant section.
Hint
Check which functions are identical, what time quantifier an IQC has, and which matrices the certificate actually covers.
Worked solution
Step 1. (i) Channels 1 and 2 have the identical shifted map $\operatorname{ReLU}(s)$; channel 3 has $\operatorname{ReLU}(s+1)-1$. At $s=-1/2$ their values are $0$ and $-1/2$. Coupling justified by repetition may be used within the equal-shift group, or replaced by per-channel diagonal constraints across all three.
Step 2. (ii) A hard IQC guarantees each finite-horizon sum is nonnegative. Individual terms may be negative. Sum the dissipation inequality with the IQC, retaining its filter state and initial-state assumptions, instead of asserting pointwise nonnegativity.
Step 3. (iii) The changed weights change the LMI and possibly local bounds/equilibria. Recompute and recheck the certificate, or establish a perturbation bound smaller than a proven feasibility margin. A small numerical update alone does not prove that the old certificate applies.
Original research exercises
Continue here when the core calculations and the certificate assumptions are clear. Use the graded questions above as a warm-up; the original derivations below remain available in full.
Key Papers
| Paper | Venue | Contribution | Why read it |
|---|---|---|---|
| Stability Analysis Using Quadratic Constraints for Systems With Neural Network Controllers (Yin, Seiler, Arcak) | IEEE TAC 67(4), 2022 | Local offset sector QCs by interval propagation; stability LMI and ellipsoidal ROA inner approximation; IQC-robust version | The template of Section 2 and of the walkthrough |
| Linear systems with neural network nonlinearities: Improved stability analysis via acausal Zames-Falb multipliers (Pauli, Gramlich, Berberich, Allgöwer) | CDC 2021 | Hard IQCs for acausal FIR Zames–Falb multipliers; repeated-nonlinearity classes; larger certified ROAs | Dynamic multipliers for NN loops, and which class is valid when |
| Offset-free setpoint tracking using neural network controllers (Pauli, Köhler, Berberich, Koch, Allgöwer) | L4DC 2021, PMLR 144 | Integral action plus slope QCs: one LMI for all references; local versions; reference governor | Structure (an integrator) doing what training cannot |
| Robustness analysis and training of recurrent neural networks using dissipativity theory (Pauli, Berberich, Allgöwer) | at – Automatisierungstechnik 70(8), 2022 | RNN robustness via dissipativity: $\mathcal H_2$ and $\ell_2$-gain measures, LMI constraints for training | From feedforward Lipschitz bounds to recurrent models |
| Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees (Junnarkar, Arcak, Seiler) | Automatica 193, 2026 | Convexifying change of variables for full-order implicit NN controllers; RL with a dissipativity projection | Synthesis with a hard closed-loop constraint |
| Learning Over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear System (Wang, Barbara, Revay, Manchester) | IEEE L-CSS 7, 2023 | Nonlinear Youla parameterisation with a REN: every parameter value gives a contracting, Lipschitz closed loop, and large enough RENs approximate, on bounded inputs, every controller that achieves such a loop | Stability by construction instead of by constraint |
| Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programming (Hu, Fazlyab, Morari, Pappas) | CDC 2020 | Reachable-set over-approximation of NN loops by an SDP of QCs, incl. coupled repeated-ReLU constraints | Safety (where trajectories go), and where coupling is valid |
| Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation (Yang, Dai, Shi, Hsieh, Tedrake, Zhang) | ICML 2024 | Verifiable ROA condition on a box; neural Lyapunov functions and observers checked by α,β-CROWN | The bound-propagation alternative for nonlinear plants |
| Discrete-Time Stability Analysis of ReLU Feedback Systems via Integral Quadratic Constraints (Vahedi Noori, Hu, Dullerud, Seiler) | arXiv 2025 | ReLU-specific hard IQCs containing the Zames–Falb class | Where multiplier design is heading |
| Scalable Incremental Robustness Analysis of Neural Network Feedback Systems (Wang, Seiler, Dullerud, Hu) | arXiv 2026 | $\Lambda$-incremental QC; decomposition making the control LMIs depth-independent | The current answer to "does it scale?" |
| System analysis via integral quadratic constraints (Megretski, Rantzer) | IEEE TAC 42(6), 1997 | The IQC framework and stability theorem | The language of Sections 2, 3 and 8 |
| Dissipativity and Integral Quadratic Constraints: Tailored Computational Robustness Tests for Complex Interconnections (Scherer) | IEEE CSM 42(3), 2022 | IQC theorems via dissipation and hard IQCs; tailored LMI tests | The proof route used by Yin et al. and Pauli et al. |
| Zames-Falb multipliers for absolute stability: From O'Shea's contribution to convex searches (Carrasco, Turner, Heath) | European J. Control 28, 2016 | Survey of Zames–Falb multipliers and convex (FIR) searches | Get the multiplier conditions right |
| Neural Networks in the Loop: Learning with Stability and Robustness Guarantees (Manchester, Wang, Barbara) | Annu. Rev. Control Robot. Auton. Syst. 9, 2026 | Survey: direct parameterisations, RENs, Youla-type policies | A map of the synthesis side of this module |