Muon for Phase Retrieval — Proof map and the hard bridge
This post is the proof spine and difficulty map. The point is not to list theorems. The point is to show which steps are exact, which steps are imported from Paquette-style resolvent theory, and which step remains the genuinely hard same-sample RMT bridge.
There are four reductions:
- Easy and exact: the quadratic loss closes on a finite frame.
- Imported but explicit: Paquette’s recursion turns fixed-(a) dynamics into Volterra.
- Algebraic: finite-rank resolvent equations turn the Volterra state into BBP roots and residues.
- Hard: a same-sample local law is needed only to transport the fresh/frozen resolvent formulas to the reused training data.
1. Finite-frame closure
Let [ Z=[W,\Theta], \qquad Z^\top Z= \begin{pmatrix} Q&M\M^\top&I \end{pmatrix}. ] For the quadratic phase-retrieval loss, the population gradient has the form [ G(W)=ZA_{\rm gd} ] for a finite matrix (A_{\rm gd}). This is the first closure: the gradient does not point outside the span of the current weights and teacher directions.
Muon applies a singular-value map. If (Y=ZA), then [ Y^\top Y=A^\top Z^\top ZA. ] Write (\mathcal G=Z^\top Z) and (\widehat A=\mathcal G^{1/2}A). The singular values of (Y) are those of (\widehat A). Therefore [ \mathsf M_a(Y) = Z\mathcal G^{-1/2}\mathsf M_a(\widehat A). ] Applying this to (G(W)) gives [ \mathsf M_a(G(W))=ZA_a. ] Split (A_a=(U_a^\top,V_a^\top)^\top). The Muon flow is [ \dot W=-\eta_a(WU_a+\Theta V_a). ] Differentiate (Q=W^\top W) and (M=W^\top\Theta): [ \dot Q = -\eta_a(U_a^\top Q+QU_a+V_a^\top M^\top+MV_a), ] [ \dot M = -\eta_a(U_a^\top M+V_a^\top). ] With (C=M^\top Q^{-1}M), differentiating (C) and substituting the two previous equations gives [ \dot C = -\eta_a\bigl[(I-C)D_a+D_a^\top(I-C)\bigr], \qquad D_a=V_aQ^{-1}M . ] This is exact. No high-dimensional limit has entered yet.
2. From mode equations to Volterra
For a fixed exponent (a), Paquette’s resolvent formulas give deterministic coefficients (\delta_i^{(a)}) and (\nu_i^{(a)}). The scalar projected mode equation is [ \dot q_i^2(t) = -2\eta\,\delta_i^{(a)}Q(t)^{(a-1)/2}q_i^2(t) + \eta^2\nu_i^{(a)}Q(t)^a . ] Change time by [ d\tau=\eta Q(t)^{(a-1)/2}dt . ] Then [ \partial_\tau q_i^2 = -2\delta_i^{(a)}q_i^2 + \eta\nu_i^{(a)}Q(\tau)^{(a+1)/2}. ] Solving this linear equation gives [ q_i^2(\tau) = e^{-2\delta_i^{(a)}\tau}q_i^2(0) + \eta\nu_i^{(a)} \int_0^\tau e^{-2\delta_i^{(a)}(\tau-u)} Q(u)^{(a+1)/2}du . ] If (Q(\tau)=\sum_iw_iq_i^2(\tau)), then [ Q(\tau)=F_a(\tau) + \eta\int_0^\tau K_a(\tau-u)Q(u)^{(a+1)/2}du, ] where [ F_a(\tau)=\sum_iw_ie^{-2\delta_i^{(a)}\tau}q_i^2(0), \qquad K_a(s)=\sum_iw_i\nu_i^{(a)}e^{-2\delta_i^{(a)}s}. ] This is the Volterra law. The only deep input is the Paquette computation of (\delta_i^{(a)}) and (\nu_i^{(a)}) from one- and two-resolvent formulas for the chosen spectral filter.
3. Outlier roots and residues
Once the Volterra state is known, the spectra are finite-rank perturbations of bulk matrices. Write one channel as [ \mathcal H= \begin{pmatrix} A&B\B^\top&C \end{pmatrix}. ] An eigenvector ((u,v)) with eigenvalue (\lambda) solves [ (A-\lambda I)u+Bv=0, \qquad B^\top u+(C-\lambda I)v=0 . ] If (\lambda\notin{\rm spec}(C)), the second equation gives [ v=(\lambda I-C)^{-1}B^\top u. ] Substitution gives the finite root equation [ \Bigl[ \lambda I-A-B(\lambda I-C)^{-1}B^\top \Bigr]u=0. ] Thus an outlier is a zero of a finite determinant. The teacher mass of the outlier is the residue of the projected resolvent. If [ S(\lambda)=A+B(\lambda I-C)^{-1}B^\top ] and (\lambda_\star) is a simple root with finite vector (u_\star), then [ \Omega_\star= \frac1{u_\star^\top(I-\partial_\lambda S(\lambda_\star))u_\star}. ] This formula is why the plots track finite roots and residues, not sorted empirical ranks.
4. The RMT bridge
For fresh or frozen samples, the Paquette and Dyson resolvent equations apply directly: the state is fixed before the resolvent sample is generated. The real same-sample issue is different. The training data also produce (W_t), and then the same data are used to form the resolvent. The desired estimate is [ \varepsilon_{\rm bridge,d} = \sup_{t,z} \left|F_d(z,t)-F_d^{\rm fr}(z,t)\right| + \left|\partial_zF_d(z,t)-\partial_zF_d^{\rm fr}(z,t)\right| = o_{\mathbb P}(1). ] Here (F_d) is the reused-data finite resolvent matrix, while (F_d^{\rm fr}) is the fresh or leave-one-out version. The derivative is included because residues depend on it.
The proof route is standard in spirit but hard in estimates:
- remove one sample and compare (W_t) with the leave-one-out trajectory (W_t^{(\ell)});
- decompose each sample into its finite projection on ([W_t,\Theta]) and an orthogonal Gaussian part;
- apply a local law to the orthogonal block, uniformly over the time grid and the spectral contours;
- smooth the dense BBP handoff so that branch labels are replaced by contour integrals, finite roots with margin, and visible masses.
Once this estimate is available, the rest is analytic. On a contour surrounding a simple deterministic root, uniform convergence of the finite matrix gives uniform convergence of the determinant. Rouche’s theorem gives one empirical root in the contour. Differentiating the same finite resolvent expression gives the residue. A positive margin from the bulk edge makes the exit time stable.
The current theory should therefore be read in two layers. The reduced layer is explicit: finite-frame Muon, Volterra learning curves, finite roots, residues and BBP times. The full same-sample theorem is the RMT transport layer. It is not a new optimization idea; it is the high-probability justification that the empirical reused-data run follows the fresh/frozen spectral readouts.