\section{Introduction}
\label{sec:introduction}

Depth can be statistically useful only if training converts composition into a
sequence of simpler estimation problems.  This principle is explicit in recent
theory for hierarchical Gaussian targets: controlled gradient descent reduces
effective dimension one layer at a time
\citep{DandiPesceZdeborovaKrzakala2025}; layerwise three-layer procedures first
recover a nonlinear feature and then fit its outer link
\citep{NichaniDamianLee2025,WangNichaniLee2023}; and a solvable spectral model
turns sharp, feature-wise transitions into an aggregate power-law learning
curve \citep{WortsmanZurichEtAl2026}.  These works explain why depth may help,
but they do not determine the joint dynamics of all matrices in a
three-layer network trained by a spectral optimizer.

The optimizer creates a second gap.  Muon and related matrix updates act on
singular values rather than coordinatewise gradient entries.  In a tractable
matrix regression model, \citet{PaquetteEtAl2026} show how SignSVD changes the
effective preconditioning and can outperform SignSGD in anisotropic regimes.
For a compositional network, however, the first feature matrix, the second
direct matrix, and the feature-mixing matrix have different dimensions and
different gradient spectra.  Transferring a scalar Muon exponent from a
two-layer model is therefore a hypothesis, not a theorem.

A third gap concerns the Hessian.  In multi-index problems, extensive
finite-sample Hessian blocks are matrix weighted sample-covariance matrices.
The global law and variational support-edge formulas for block-Wishart matrices
are now available \citep{MontanariSaeed2026}.  A support edge is not yet a BBP
theorem: outlier locations and eigenvector overlaps require anisotropic
resolvent control, and a training Hessian evaluated on reused observations
requires a dynamic leave-one-out argument.  Polynomial activations introduce
an additional tail problem because the matrix weights are unbounded.

The surrounding literature also studies genuinely different ensembles.
Classical BBP and finite-rank deformation theory concern a prescribed spike
and random bath \citep{BaikBenArousPeche2005,BenaychGeorgesNadakuditi2011}.
Kac--Rice theory counts critical points conditioned on energy and overlaps
\citep{MaillardBenArousBiroli2020,MontanariSaeedTopological2026}, while
dynamic studies follow a Hessian or another spectral observable along
gradient flow
\citep{BonnaireBiroliCammarota2025,CoeurdouxFerreBouchaud2026}.  None of
these laws is automatically the same as a fresh Hessian conditional on a
trained state.  We make the probability-space dictionary explicit in
\Cref{sec:kac-rice-interface}, rather than using ``BBP'' as a visual synonym
for any moving extreme eigenvalue.

This paper places those three gaps in one model while keeping their proof
levels separate.  We study a two-hidden-block quadratic residual architecture,
called three-layer because the trainable maps are the first feature map, the
second feature map, and the scalar readout.  In latent teacher coordinates
\(z\in\R^k\), the network is
\begin{align}
 h &= Vz+b, &
 q &= \He_2(h)=h^{\odot2}-\one, \nonumber\\
 r &= Uz+Aq+c, &
 s &= \He_2(r)=r^{\odot2}-\one, \label{eq:intro-network}\\
 f_\theta(z)&=a_0+\alpha^\trans q+\beta^\trans s. \nonumber
\end{align}
It is nonlinear in the parameters but only degree four in \(z\).  This fact
gives an exact finite-dimensional population theory without invoking a width
or ambient-dimension limit.

\subsection{Contributions}

Our contributions are the following.

\paragraph{Exact deterministic population theory.}
We prove that \eqref{eq:intro-network} has a finite multivariate Hermite
expansion through degree four.  For a degree-four teacher, population risk,
gradient flow, and the complete Hessian are exact algebraic functions of the
coefficient map.  Tensor Gauss--Hermite quadrature with at least five nodes per
coordinate evaluates every coefficient and the risk exactly.  This is stronger
than a Monte Carlo approximation but should not be called a high-dimensional
DMFT.

\paragraph{A rigorous sectoral hierarchy.}
Weighted degree-wise coefficient Jacobians define Gram blocks for degrees
\(2,3,4\).  We identify their Schur complements with squared residualized
Jacobians.  These matrices are always positive semidefinite and measure the
instantaneous parameter directions that a new Hermite sector adds after the
earlier sectors have been projected out.  Combining them with the exact
residual-curvature Hessian gives a checkable lower bound on active curvature.

\paragraph{Muon geometry without an unjustified exponent.}
For a matrix gradient \(G\), we analyze the same normalized singular-value
power map used in the code.  Polar Muon is the \(a=0\) member, while \(a=1\)
recovers the gradient singular-value profile up to normalization.  We prove a
strict first-order descent identity and a sufficient smooth-step bound for all
\(a\geq0\).  The result licenses the family as an optimizer; it does not select
one common \(a\) for \(V,U,A\).  We derive its exact matrix Fr\'echet
derivative, which exposes the response contraction required by an Onsager
term.

\paragraph{Exact fresh-sample block-Wishart lift.}
After embedding a reduced state in \(\R^d\), we derive the analytic
preactivation Hessian.  On a fresh Gaussian sample, the orthogonal direct-weight
block is conditionally
\[
 \frac1n\sum_{\mu=1}^n T_\theta(z_\mu)\kron
 \xi_\mu\xi_\mu^\trans,
\]
with \(\xi_\mu\) independent of the fixed-size self-adjoint block
\(T_\theta(z_\mu)\).  We may therefore apply the Montanari--Saeed global
block-Wishart theory after bounded spectral clipping.  An exact finite Schur
determinant identifies the equation that would become a dynamic BBP condition
once an anisotropic local law is proved.

\paragraph{A DMFT and scaling-law matching program.}
We formulate the minimum two-time covariance and response kernels required by
joint, reused-data training.  The frozen-feature specialization matches the
random-feature DMFT of \citet{KrampLindnerHelias2026}; the population
anisotropic limit matches the three-phase Volterra perspective of
\citet{BraunLoureiroMinhImaizumi2025}; and the feature-wise sample threshold
matches the sequential-recovery mechanism of
\citet{WortsmanZurichEtAl2026}.  The joint closure itself is explicitly stated
as a conjecture.  We distinguish its MSRJD, dynamic-cavity, replica, and
spectral-AMP formulations.  In particular, Muon-\(a\) supplies a
nonseparable spectral channel but becomes an AMP algorithm only after a
self-consistent matrix Onsager reaction is derived.

\paragraph{Controlled extension beyond quadratic activations.}
For an arbitrary twice differentiable coefficient map into Gaussian
\(L^2\), we prove nonasymptotic Hermite-tail bounds for population risk,
gradient, and Hessian.  The quadratic model is the zero-tail case.  This
separates the algebra that extends to smooth links from the local laws,
moment estimates, and reused-data closure that must be reproved for each
high-dimensional model.

\paragraph{Reproducible empirical stress test.}
We compare gradient descent, exact polar Muon, and powers \(1/7\) and \(1/3\)
under paired initializations and datasets.  We evaluate population risk exactly
and keep raw polynomial Hessians separate from clipped, theorem-compatible
ones.  A three-pair pilot suggests an endpoint gain, but a pre-specified
twenty-pair confirmation does not reproduce it.  Instead it reveals earlier
sector half-error clocks with no uniform improvement in event coverage or
terminal risk.  The fixed latent rank remains too small for a scaling exponent,
and the spectral panel is exploratory.

\subsection{Relationship to the two-layer theory}

The two-layer phase-retrieval program closes on low-rank overlap matrices and,
under anisotropy, on a Volterra/Stieltjes hierarchy.  In the present model there
are three distinct obstructions to copying that calculation.
\begin{enumerate}[leftmargin=*]
  \item The feature basis itself changes: \(q\) depends on \(V\), while \(s\)
  depends jointly on \(V,U,A\).
  \item The coefficient map includes degrees \(2,3,4\), so the active tangent
  space grows by residualized sectors rather than a single overlap.
  \item The empirical direct-weight Hessian has block size \(p+q\) and is
  coupled through a finite feature-mixing/readout sector.
\end{enumerate}
Our exact coefficient and Schur formulas are the three-layer replacements for
the two-layer overlap closure.  The fresh block-Wishart identity is the
three-layer replacement for the scalar weighted-Wishart bulk.  What remains
open is precisely the joint dynamic local law, rather than an unspecified
``DMFT correction.''

\subsection{Organization}

\Cref{sec:model} defines the model and probability spaces.
\Cref{sec:hermite} gives the exact population closure.
\Cref{sec:sectors} develops the sectoral Schur hierarchy.
\Cref{sec:muon} analyzes the spectral update.
\Cref{sec:rmt} proves the fresh-sample block-Wishart identity and finite Schur
reduction.  \Cref{sec:dmft} states the high-dimensional matching program.
\Cref{sec:universality} proves the Hermite-tail transfer and states the
conditional DMFT--RMT--BBP closure.
\Cref{sec:experiments} reports the checked-in experiments, and
\Cref{sec:discussion} gives the proof-status audit and open problems.
