\section{DMFT matching and scaling-law predictions}
\label{sec:dmft}

This section is deliberately split between an exact finite-row identity and a
conjectural reused-data mean-field closure.

\subsection{Exact finite-row representation of the spectral map}

\begin{proposition}[Muon as a finite Gram action]
\label{prop:finite-muon-action}
Let \(G\in\R^{m\times d}\), \(m\leq d\), have full row rank, and ignore the
numerical singular-value floor.  Then the unnormalized singular-value power
map has the exact representation
\begin{equation}
 L\Diag((\sigma_i/\bar\sigma)^a)R^\trans
 =\bar\sigma^{-a}(GG^\trans)^{(a-1)/2}G.
 \label{eq:finite-muon-action}
\end{equation}
For rank-deficient \(G\), the same identity holds on \(\Ran(G)\) with the
Moore--Penrose functional calculus.  The unit-RMS Muon map differs only by a
scalar normalization.
\end{proposition}

Thus, when \(p,q\) remain fixed while \(d\to\infty\), Muon on \(W_1\) and
\(W_2\) depends on finite \(p\times p\) and \(q\times q\) gradient Gram
matrices.  This is the cleanest match to the finite-overlap reduction in the
two-layer theory.  The difficulty is not the spectral functional calculus; it
is closing the stochastic evolution of those Gram matrices jointly with
\(A,\alpha,\beta\) and the reused sample.

\subsection{Candidate low-rank DMFT state}

Consider \(n,d\to\infty\) with \(n/d\to\alpha\), fixed \(k,p,q\), and direct
rows
\[
 w_a(t)\in\R^d,\qquad a=1,\dots,p+q,
\]
obtained by stacking the rows of \(W_1(t)\) and \(W_2(t)\).  A minimal dynamic
state contains:
\begin{align}
 Q_{ab}(t,s)&=w_a(t)^\trans w_b(s),
 \label{eq:dmft-Q}\\
 M_{aj}(t)&=w_a(t)^\trans\Theta_j,\qquad j=1,\dots,k,
 \label{eq:dmft-M}\\
 R_{ab}(t,s)&=\frac1d\sum_{i=1}^d
 \frac{\delta w_{a i}(t)}{\delta \zeta_{b i}(s)},
 \label{eq:dmft-R}
\end{align}
where \(\zeta\) is an infinitesimal external field.  The finite parameters
\[
 \varphi(t)=(A(t),b(t),c(t),\alpha(t),\beta(t),a_0(t))
\]
must remain in the state because they determine the link derivatives
\eqref{eq:link-gradient}--\eqref{eq:T-block}.

For fresh online samples, one expects effective Gaussian preactivations whose
two-time covariance is determined by \(Q\), coupled to a finite deterministic
evolution of \((M,\varphi)\).  For repeated passes over a fixed dataset, the
effective process additionally contains an Onsager memory term built from
\(R\) and colored noise whose covariance depends on the two-time residual
correlations.

\begin{conjecture}[Joint three-layer reused-sample DMFT]
\label{conj:joint-dmft}
Under Gaussian covariates, fixed bottleneck dimensions, proportional sample
size, exchangeable initialization, and a regularized Lipschitz spectral map,
the empirical trajectory of the finite observables
\[
 \big(Q(t,s),M(t),R(t,s),\varphi(t)\big)
\]
converges on every fixed time horizon to a deterministic causal solution.  A
representative training example is described by an effective non-Markovian
process
\begin{equation}
 h(t)=h^{\mathrm{sig}}(M(t),Z)
 +\eta(t)
 +\int_0^t\Sigma_R(t,s)\,
 \cG(h(s),\varphi(s),Y)\,\dd s,
 \label{eq:effective-process}
\end{equation}
where \(\eta\) is centered colored Gaussian noise, its covariance
\(\Sigma_C\) is a functional of \(Q\) and residual correlations, and
\(\Sigma_R\) is a response kernel.  The matrix drifts for the direct blocks
are obtained by applying the finite action
\eqref{eq:finite-muon-action} to their limiting gradient Gram matrices.
\end{conjecture}

This conjecture specifies the state and causal structure but not the final
self-consistency equations.  A proof requires dynamic cavity or generating
functional methods, tightness, and uniqueness.  The existing code does not
claim to have supplied these steps.

\subsection{MSRJD construction: the physics target}

The appropriate statistical-physics derivation is a disorder-averaged
generating functional, not a Gaussian ansatz imposed directly on the
parameters.  In a continuous-time regularization of the training rule, stack
the direct rows in \(W(t)\) and write schematically
\begin{equation}
 \dot W(t)=-\cP_a\!\left(G_W(t)\right)+\zeta(t),
 \label{eq:regularized-muon-flow}
\end{equation}
where \(G_W\) is the fixed-sample gradient and \(\cP_a\) is the regularized
singular-value map.  Introducing response fields \(\widehat W\) gives an
MSRJD functional of the form
\begin{equation}
 \cZ[\zeta]=\int\cD W\,\cD\widehat W\,
 \exp\!\left\{
 -\int\dd t\,
 \left\langle\widehat W(t),
 \dot W(t)+\cP_a(G_W(t))-\zeta(t)\right\rangle
 \right\},
 \label{eq:msrjd-schematic}
\end{equation}
with the usual response-field contour convention suppressed.  The
Martin--Siggia--Rose--De Dominicis--Janssen construction and its effective
action are the standard field-theoretic route to dynamic mean-field theory
\citep{HeliasDahmen2020}.  Averaging \eqref{eq:msrjd-schematic} over the
reused Gaussian training sample creates nonlocal temporal interactions.

At the saddle, the required collective fields are precisely the two-time
correlations and responses \(Q,R\), the teacher overlaps \(M\), and the finite
state \(\varphi\), together with pattern-level preactivation fields for both
quadratic blocks.  The latter are essential: the same training pattern enters
\(h=Wx\), the nonlinear features, the residual, and every later gradient.
After disorder averaging, this repeated use becomes a retarded self-interaction
and colored effective noise.  The exact finite Gram action
\eqref{eq:finite-muon-action} shows that Muon does not introduce an extensive
new order parameter at fixed row dimension; it changes the finite matrix
functional applied to the saddle-point gradient Gram matrices.

This construction follows the field-theory lineage used to derive Gaussian
process saddles and systematic finite-width corrections for deep and
recurrent networks \citep{SegadloEtAl2022}.  Here, however, the saddle is
dynamic and task conditioned: its teacher overlaps and coefficient map move
during training.

\subsection{Cavity, replica, and the spectral-AMP interpretation}

The three standard disordered-system routes answer different questions and
should agree where their domains overlap.

\paragraph{Dynamic cavity.}
Remove one training pattern \(\mu\), run the trajectory in the cavity system,
and expand the full trajectory in the influence of that pattern.  At leading
order one expects a relation of the schematic form
\begin{equation}
 h_\mu(t)
 =h_\mu^{\mathrm{cav}}(t)
 +\int_0^t\cO(t,s)\,
 g_\mu\big(h_\mu^{\mathrm{cav}}(s),\varphi(s)\big)\,\dd s
 +o_d(1),
 \label{eq:dynamic-cavity-reaction}
\end{equation}
where \(\cO\) is the Onsager reaction kernel.  In a reused-data problem this
reaction is generically retarded rather than a one-step scalar.  Establishing
\eqref{eq:dynamic-cavity-reaction} uniformly over patterns and time is the
leave-one-out part of the joint DMFT proof.

\paragraph{Replica.}
For a Langevin or Bayesian equilibrium, the static object is
\[
 Z_\beta(\cD)=\int\exp[-\beta\cL_\cD(\theta)]\,\dd P_0(\theta),
 \qquad
 \E_\cD\log Z_\beta
 =\lim_{r\downarrow0}\frac{\E_\cD Z_\beta^r-1}{r}.
\]
A replica-symmetric saddle would give a static potential in overlap and
representation order parameters.  It can locate information-theoretic and
metastability transitions and should agree with stationary points of a valid
Bayes-matched state evolution.  The TAP/Onsager viewpoint also admits direct
iterative constructions in canonical spin-glass models
\citep{Bolthausen2014}.  It does not determine the transient dynamics of
deterministic Muon, and replica symmetry may fail.  The Low-RAMP
literature makes the AMP/TAP/replica correspondence explicit for low-rank
matrix estimation \citep{LesieurKrzakalaZdeborova2017}; the present
hierarchical factor graph is not one of the models solved there.

\paragraph{Muon-\(a\) as an AMP-like spectral channel.}
At fixed row dimension, \eqref{eq:finite-muon-action} writes the update as a
nonseparable spectral map of a finite Gram matrix.  In that precise sense,
Muon-\(a\) is a natural matrix denoiser or channel for an AMP construction.
Rigorous scalar state evolution begins with dense Gaussian AMP, and extends
to nonseparable Lipschitz denoisers
\citep{BayatiMontanari2011,BerthierMontanariNguyen2020}.  VAMP extends scalar
state evolution to right-orthogonally invariant linear operators
\citep{RanganSchniterFletcher2019}.

The optimizer tested in \Cref{sec:experiments} is nevertheless \emph{not} an
AMP algorithm: it applies \(\Phi_a\) to the ordinary reused-data gradient and
contains no explicit Onsager correction.  A candidate long-memory
spectral-AMP iteration would instead have the structure
\begin{align}
 G_{B,\mathrm{cav}}^t
 &=G_B^t-\sum_{s<t}\Omega_B(t,s)\,\Delta B^s,
 \label{eq:spectral-amp-cavity-gradient}\\
 B^{t+1}
 &=B^t-\eta_B^t\,
 \Phi_{a_B^t,\eps}\!\left(G_{B,\mathrm{cav}}^t\right),
 \qquad B\in\{V,U,A\},
 \label{eq:spectral-amp-update}
\end{align}
where the matrix memory kernels \(\Omega_B\) are fixed by response
self-consistency and contractions of
\(D\Phi_{a_B,\eps}\), whose exact full-rank formula is
\eqref{eq:muon-frechet-normalized}.  Because the same observations pass
through two nonlinear blocks, a scalar one-step Onsager coefficient is not
assumed to suffice.

\begin{conjecture}[Spectral-AMP state evolution]
\label{conj:spectral-amp}
There exists a regularized, blockwise, long-memory AMP version of
\eqref{eq:spectral-amp-update} whose finite-time state evolution closes on
\((Q,M,R,\varphi)\) and the two layerwise effective fields.  Its Onsager
kernels cancel the leading reused-pattern self-interaction, and its
state-evolution fixed points coincide with replica-symmetric stationary
conditions whenever the latter are stable.
\end{conjecture}

Proving this conjecture requires more than observing Gaussian gradient
histograms: one must derive the factor graph, the matrix-valued Onsager
reaction, and a state evolution for the nonseparable spectral channel.

\subsection{Lazy/kernel control and rich feature-learning target}

Three limits constrain any valid closure.

\paragraph{Frozen features: the lazy/kernel control.}
If \(V,U,A\) are frozen and only a linear readout is trained, the microscopic
feature modes decouple conditionally on common response and noise kernels.  The
closure must reduce to the random-feature DMFT of
\citet{KrampLindnerHelias2026}.  Their microscopic model is linear regression
in a fixed Gaussian feature basis.  It describes NNGP, NTK, and random-feature
limits in which the representation does not acquire an \(O(1)\)
task-dependent deformation.  Its disorder average yields independent
eigenmode processes coupled through a common response kernel and colored
noise.  A proposed three-layer DMFT that does not reproduce this memory and
noise after freezing \(V,U,A\) is inconsistent.  Conversely, copying that
closure while allowing only the readout to move would remove the phenomenon
studied here.

\paragraph{Non-lazy equilibrium: the representation control.}
The Bayesian mean-field theory of \citet{VanMeegenSompolinsky2025} shows that
wide networks in the non-lazy scaling can acquire \(O(1)\), task-dependent
representations and that their coding structure depends strongly on the
activation: linear, sigmoidal, and ReLU links produce qualitatively different
codes.  This is a static posterior theory rather than a reused-data training
DMFT, but it supplies a second consistency check.  A rich-regime closure must
retain activation-dependent single-site laws rather than close only on the
kernel.  \Cref{thm:hermite-tail-transfer} gives the corresponding finite
population control in the present coefficient language.

\paragraph{Population anisotropy.}
With fresh population gradients and power-law input covariance, a quadratic
single-index specialization must reproduce the infinite weighted-overlap
hierarchy and Duhamel--Volterra phases of
\citet{BraunLoureiroMinhImaizumi2025}: fast escape, macroscopic convergence,
and spectral-tail learning.  In the isotropic finite-\(k\) reduced model,
\Cref{thm:exact-closure} replaces that infinite hierarchy by the exact finite
Hermite coefficient flow.

\subsection{A longitudinal signal coupled to a transverse bath}

The block decomposition \eqref{eq:hessian-block} has a direct physical
interpretation.  The extensive orthogonal block \(B_n(t)\) is a transverse
spectral bath.  The finite teacher and feature-mixing block \(D_n(t)\) is the
longitudinal signal sector, and \(C_n(t)\) couples the two.  Integrating out
the bath produces the exact, energy-dependent self-energy
\begin{equation}
 \Sigma_n(t,\lambda)
 =C_n(t)^\trans(B_n(t)-\lambda\Id)^{-1}C_n(t),
 \label{eq:spectral-self-energy}
\end{equation}
so the signal propagator is controlled by
\[
 D_n(t)-\lambda\Id-\Sigma_n(t,\lambda).
\]
This is precisely the finite Schur determinant \eqref{eq:Fn}.  In a valid
high-dimensional closure, the DMFT single-site process determines the
equal-time law \(\nu_t\) of the Hessian block \(T_t\); the matrix Dyson
equation turns \(\nu_t\) into the transverse bath spectrum; and the
anisotropic resolvent determines the self-energy seen by a teacher-aligned
longitudinal mode.

A BBP transition is then a stability change of this longitudinal mode against
the transverse continuum.  Below threshold the Schur zero is absorbed by the
bath edge and its eigenvector weight is delocalized.  Above threshold a
separated zero survives with nonvanishing teacher residue.  This physical
picture is exact at the level of the finite bath integration, but its
deterministic dynamic limit still requires the local-law and data-reuse locks
listed in \Cref{sec:discussion}.

\subsection{The solvable transient-BBP control}
\label{sec:solvable-transient-bbp}

The closest closed dynamical control is the linear teacher--student model of
\citet{CoeurdouxFerreBouchaud2026}.  Its gradient flow is an explicit matrix
semigroup.  A two-block input covariance produces a time-dependent
Wigner-type bath described by a \(2\times2\) Dyson equation, while the
rank-one teacher becomes a time-dependent rank-two perturbation after
symmetrization.  The associated \(2\times2\) determinant yields three
regimes: no separated branch, a persistent branch, or a branch that separates
for a finite time window and is then reabsorbed by the bulk.  This is a
genuine transient BBP mechanism and a useful null model for early stopping.

It is not a special case of the Hessian ensemble in this paper.  The spectral
observable there is a symmetrized trained weight matrix with a Wigner-type
noise component; here it is a Hessian whose transverse block is
matrix-weighted Wishart.  The methodological correspondence is
\begin{center}
\small
\begin{tabular}{p{0.20\linewidth}p{0.29\linewidth}p{0.38\linewidth}}
\toprule
Ingredient & Solvable linear control & Three-layer target\\
\midrule
training state & explicit semigroup & MSRJD/cavity state \((Q,M,R,\varphi)\)\\
spectral bath & two-block Wigner type & matrix-weighted block Wishart\\
bulk equation & \(2\times2\) Dyson equation & matrix Dyson equation driven by \(\nu_t\)\\
signal sector & time-dependent rank two & teacher coordinates plus finite parameters\\
outlier test & \(2\times2\) determinant & finite Schur determinant \(F_n\)\\
main obstruction & explicit coefficients & moving representation, reuse, tails, Muon response\\
\bottomrule
\end{tabular}
\end{center}
The absent/persistent/transient trichotomy is therefore a qualitative null
hypothesis for the modes in \Cref{fig:master-mechanism}, not a label that can
be assigned to them from a finite plot.

\subsection{Kac--Rice, landscape complexity, and BBP}
\label{sec:kac-rice-interface}

Kac--Rice and DMFT condition on different objects.  If
\(\mathcal N_d(e,q)\) counts critical points at risk density \(e\) and teacher
overlap \(q\), the landscape observables are the annealed and quenched
complexities
\[
 \Sigma_{\mathrm{ann}}(e,q)
 =\lim_{d\to\infty}\frac1d\log\E\mathcal N_d(e,q),
 \qquad
 \Sigma_{\mathrm{que}}(e,q)
 =\lim_{d\to\infty}\frac1d\E\log\mathcal N_d(e,q).
\]
The Kac--Rice identity itself is exact under regularity assumptions
\citep{Kac1943,Rice1945,AdlerTaylor2007}.  Evaluating the quenched complexity,
or replacing it by an annealed calculation, is a separate step.  Random
matrix conditioning of the Hessian has made this route effective in spin
glasses and non-Gaussian empirical-risk landscapes
\citep{AuffingerBenArousCerny2013,MaillardBenArousBiroli2020}.

Recent single-index results sharpen the interface relevant here.
\citet{AsgariMontanariSaeed2025} establish general high-dimensional
Kac--Rice formulas for local minima under explicit regularity and
concentration hypotheses.
\citet{MontanariSaeedTopological2026} prove a finite-dimensional variational
formula for the annealed exponential rate of local minima and sufficient
conditions for rate trivialization.  \citet{MaillardBonnaireBiroli2026}
condition further on overlap and combine Kac--Rice with a low-rank Hessian
analysis: a teacher-aligned BBP instability can appear before the annealed
complexity of minima vanishes.  At initialization and along phase-retrieval
gradient flow, related studies find continuous or discontinuous informative
Hessian branches and strong finite-size effects
\citep{AnnesiBocchiCammarota2025,BonnaireBiroliCammarota2025}.

These results do not turn the fresh Hessian of \Cref{sec:rmt} into a
Kac--Rice Hessian.  Four probability laws must be kept distinct:
\begin{enumerate}[leftmargin=*]
 \item the population Hessian at a prescribed parameter;
 \item the fresh empirical Hessian conditional on a trained state;
 \item the same-data Hessian along a random training trajectory;
 \item the Hessian at a critical point conditioned on risk, overlaps, and
 representation invariants.
\end{enumerate}
The fourth object requires a new three-layer Kac--Rice calculation.  It must
remove gauge zero modes, condition the joint gradient/Hessian law on all
layerwise overlaps, evaluate the exponential determinant, and distinguish
annealed from quenched complexity.  Such a calculation would map the
stationary topology.  It would still not replace the causal response kernels
needed for transient Muon dynamics.

\paragraph{Continuous versus discontinuous detachment.}
The classical BBP transition and general finite-rank deformation theory
identify the continuous separation of a spike from a regular edge
\citep{BaikBenArousPeche2005,BenaychGeorgesNadakuditi2011}.  A different
mechanism occurs when the limiting density vanishes sufficiently rapidly at
the relevant edge: the asymptotic eigenvector overlap can jump, with an
extended finite-size precursor region
\citep{BocchiBiroliCammarotaRicciTersenghi2026}.  Whether a three-layer
Hessian contact is continuous, discontinuous, or multicritical cannot be
decided from an outlier count.  It requires the edge exponent, the derivative
of the limiting Schur determinant, and the finite-size scaling of both the
eigenvalue gap and teacher residue.

\paragraph{Sparse cavity as a computational control.}
For locally tree-like, non-Hermitian matrices, cavity recursions and population
dynamics can recover spectral support and outliers
\citep{RogersCastilloKuhnTakeda2008,NeriMetz2012,NeriMetz2016}.
\citet{GiammancoValigiCammarota2026} use this machinery for sparse
antagonistic matrices with diagonal disorder and a Jacobian-like structure,
finding five spectral phases and also a strong-disorder regime where the
numerical population dynamics underestimates the support.  This is relevant
as a computational warning and as a template for distribution-valued
resolvent order parameters.  It is not an imported theorem for the dense
Hermitian block-Wishart bath considered here.

\subsection{Sequential BBP thresholds and aggregate scaling}

Suppose a lifted hierarchical stage contains orthogonal latent features with
strengths
\[
 \lambda_i\asymp i^{-\gamma},\qquad \gamma>\frac12,
\]
and effective ambient feature dimension \(D\asymp d^{q_0}\).  A spectral spike
of strength \(\lambda_i\) becomes visible when its signal exceeds the
sample-covariance fluctuation scale, giving the feature-wise prediction
\begin{equation}
 n_i\asymp\frac{D}{\lambda_i^2}
 \asymp d^{q_0}i^{2\gamma}.
 \label{eq:feature-threshold}
\end{equation}
Consequently the number of recovered features at sample size \(n\) is
\begin{equation}
 m(n)\asymp\left(\frac{n}{d^{q_0}}\right)^{1/(2\gamma)}.
 \label{eq:number-recovered}
\end{equation}
If prediction error is the unrecovered squared-strength tail, then
\begin{equation}
 \sum_{i>m(n)}\lambda_i^2
 \asymp m(n)^{1-2\gamma}
 \asymp
 \left(\frac{n}{d^{q_0}}\right)^{-1+1/(2\gamma)}.
 \label{eq:aggregate-scaling}
\end{equation}
These are the sharp-threshold and aggregate-scaling mechanisms proved in the
solvable hierarchical model of \citet{WortsmanZurichEtAl2026}, up to its precise
normalizations.

\begin{conjecture}[Hierarchical feature-front scaling]
\label{conj:feature-front}
For a growing-rank version of \eqref{eq:network} with power-law teacher
strengths, the dynamic Schur sectors and the fresh Hessian Schur determinant
describe the same feature front: feature \(i\) enters the residualized tangent
space when a finite-signal branch crosses the appropriate block-Wishart edge.
Aggregating these crossings yields \eqref{eq:aggregate-scaling}, with a
block- and depth-dependent effective dimension \(D\).
\end{conjecture}

The current experiment keeps \(k=3\).  It can test ordering and finite-size
smoothing, but no fit from six sample sizes at fixed rank can prove
\eqref{eq:aggregate-scaling}.

\subsection{Three matched phases}

The references suggest the following falsifiable dictionary:
\begin{center}
\small
\begin{tabular}{p{0.17\linewidth}p{0.23\linewidth}p{0.23\linewidth}p{0.20\linewidth}}
\toprule
Phase & Exact three-layer observable & RMT observable & Expected phenomenon\\
\midrule
Escape & \(E_2\), \(S_2\), first-layer overlap & first teacher-aligned branch &
recovery of strongest latent subspace\\
Composition & \(E_3\), \(S_3\), evolution of \(A\) &
additional finite Schur zeros & nonlinear feature mixing\\
Tail/late sector & \(E_4\), \(S_4\), weak singular channels &
late BBP contacts near moving edges & slow weak-feature recovery\\
\bottomrule
\end{tabular}
\end{center}
This table is a research hypothesis.  The experiments should compare the event
times quantitatively rather than infer causality from visually aligned curves.
