\section{Model, notation, and three probability spaces}
\label{sec:model}

\subsection{Multivariate Hermite basis}

Let \(Z=(Z_1,\dots,Z_k)\sim\mathcal N(0,\Id_k)\).  We use the probabilists'
Hermite polynomials, determined by
\[
 \exp(tx-t^2/2)=\sum_{m\geq0}\He_m(x)\frac{t^m}{m!}.
\]
For a multi-index \(\gamma=(\gamma_1,\dots,\gamma_k)\in\N^k\), write
\[
 |\gamma|=\sum_{j=1}^k\gamma_j,\qquad
 \gamma!=\prod_{j=1}^k\gamma_j!,\qquad
 \He_\gamma(z)=\prod_{j=1}^k\He_{\gamma_j}(z_j).
\]
Gaussian orthogonality gives
\begin{equation}
 \E[\He_\gamma(Z)\He_\eta(Z)]
 =\gamma!\,\mathbf1_{\{\gamma=\eta\}}.
 \label{eq:hermite-orthogonality}
\end{equation}
If \(g\in L^2(\mathcal N(0,\Id_k))\), its coefficient convention is
\begin{equation}
 c_\gamma(g)=\frac1{\gamma!}\E[g(Z)\He_\gamma(Z)].
 \label{eq:coefficient-definition}
\end{equation}

\subsection{Three-layer quadratic residual network}

Fix latent dimension \(k\), first width \(p\), and second width \(q\).  The
parameters are
\[
 \theta=(V,b,U,A,c,\alpha,\beta,a_0)
\]
with
\[
 V\in\R^{p\times k},\quad b,\alpha\in\R^p,\quad
 U\in\R^{q\times k},\quad A\in\R^{q\times p},\quad
 c,\beta\in\R^q,\quad a_0\in\R.
\]
For \(z\in\R^k\), define
\begin{align}
 h_\theta(z)&=Vz+b,&
 q_\theta(z)&=h_\theta(z)^{\odot2}-\one_p, \label{eq:first-block}\\
 r_\theta(z)&=Uz+Aq_\theta(z)+c,&
 s_\theta(z)&=r_\theta(z)^{\odot2}-\one_q, \label{eq:second-block}\\
 f_\theta(z)&=a_0+\alpha^\trans q_\theta(z)
                    +\beta^\trans s_\theta(z). \label{eq:network}
\end{align}
The untied model treats \(U\) and \(V\) independently.  A tied control may set
\(U=V\) when \(p=q\), but none of the exact coefficient identities requires
tying.

The teacher is
\begin{equation}
 f_\star(z)=\sum_{|\gamma|\leq4}c_{\star,\gamma}\He_\gamma(z).
 \label{eq:teacher}
\end{equation}
The experiments set degrees \(0,1\) to zero and include diagonal and mixed
coefficients in degrees \(2,3,4\).  We keep the general degree-four notation
because biases can generate lower student sectors.

\begin{definition}[Population risk]
The deterministic population objective is
\begin{equation}
 \cR(\theta)
 =\frac12\E\big[(f_\theta(Z)-f_\star(Z))^2\big].
 \label{eq:population-risk}
\end{equation}
\end{definition}

\subsection{Parameter and coefficient coordinates}

Let
\[
 \Gamma_4=\{\gamma\in\N^k:|\gamma|\leq4\},\qquad
 N_4=|\Gamma_4|=\binom{k+4}{4}.
\]
Order \(\Gamma_4\) once and identify the coefficient map
\[
 c(\theta)=(c_\gamma(f_\theta))_{\gamma\in\Gamma_4}\in\R^{N_4}.
\]
Set
\[
 D=\Diag((\gamma!)_{\gamma\in\Gamma_4}),\qquad
 e(\theta)=c(\theta)-c_\star,\qquad
 J(\theta)=\nabla_\theta c(\theta)\in\R^{N_4\times P},
\]
where \(P=pk+p+qk+qp+q+p+q+1\) in the untied model.  The orientation is chosen
so that \(J_{\,\gamma,j}=\partial c_\gamma/\partial\theta_j\).

For degree \(r\), let \(I_r=\{\gamma\in\Gamma_4:|\gamma|=r\}\), let
\(J_r\) be the corresponding rows of \(J\), and let \(D_r\) be the restriction
of \(D\).  The weighted parameter-space Jacobian is
\begin{equation}
 M_r=J_r^\trans D_r^{1/2}\in\R^{P\times |I_r|}.
 \label{eq:weighted-jacobian}
\end{equation}

\subsection{Population, fixed-sample training, and fresh-sample spectroscopy}

We distinguish three experiments.

\paragraph{Population dynamics.}
The expectation in \eqref{eq:population-risk} is evaluated exactly and the
parameter trajectory is deterministic after initialization.

\paragraph{Fixed-sample training.}
Given a training set \(\cD_n=\{(z_\mu,y_\mu)\}_{\mu=1}^n\), the empirical risk
is
\begin{equation}
 \widehat\cR_n(\theta)
 =\frac1{2n}\sum_{\mu=1}^n(f_\theta(z_\mu)-y_\mu)^2.
 \label{eq:empirical-risk}
\end{equation}
The same sample is reused over optimization steps.  This creates temporal
dependence and is the setting that ultimately needs a joint DMFT or dynamic
leave-one-out proof.

\paragraph{Fresh-sample spectroscopy.}
At a fixed checkpoint \(\theta_t\), draw observations independent of the
training history and form a Hessian.  Conditional on \(\theta_t\), this is a
static random-matrix problem.  The fresh construction is what makes the exact
block-Wishart reduction in \Cref{sec:rmt} possible.

\begin{remark}[Why the distinction matters]
A population ODE cannot prove finite-sample generalization.  A fresh-sample
global spectral law cannot prove the spectrum of a reused-data training
Hessian.  A support-edge theorem cannot by itself prove a BBP outlier or its
eigenvector overlap.  We use a separate notation and claim level for each
object.
\end{remark}

\subsection{Ambient embedding}

For the random-matrix lift, fix \(d>k\) and an isometry
\(\Theta\in\R^{d\times k}\), \(\Theta^\trans\Theta=\Id_k\).  Complete it to an
orthogonal matrix \([\Theta,\Theta_\perp]\).  Draw
\[
 X=\Theta Z+\Theta_\perp\Xi,\qquad
 Z\sim\mathcal N(0,\Id_k),\quad
 \Xi\sim\mathcal N(0,\Id_{d-k}),\quad Z\perp\Xi.
\]
Embed the direct matrices by
\[
 W_1=V\Theta^\trans,\qquad W_2=U\Theta^\trans.
\]
Then \(W_1X=VZ\) and \(W_2X=UZ\); the network output depends on \(X\) only
through \(Z\), while the direct-weight Hessian still has \(d\) columns per
hidden unit.  This split produces an extensive orthogonal bulk and a
fixed-dimensional signal sector.
