Janis AIAD

Low-rank neural networks are usually sold as smaller, compressed and pruned models. This paper asks a sharper question: when does low rank still preserve the optimization geometry that makes wide networks trainable? We answer this through the finite-width neural tangent kernel. Low-rank NTK training still carries an interpretable bottleneck-feature geometry, because rank controls which feature maps enter the tangent kernel. Our main result is that low-rank training is governed by a genuine three-way compromise between width, depth, and rank. To the best of our knowledge, this is the first NTK analysis showing that an optimization certificate can impose a depth-rank tradeoff of this form. In particular, for low-rank bottleneck dynamics, preserving the NTK spectral margin has a proven conservative cubic-depth sufficient rule, r ≳L^3, up to dataset-size and logarithmic factors. From the finite-network experiments, we conjecture an effective L^3/2 law for scalar-output contracted cumulants. We also give a full-rank-compatible parametrization, separating true low-rank effects from ordinary finite-width effects. True finite-network NTK experiments confirm the exact full-rank matching, the predicted operator scaling, the rank-depth tradeoff, and the finite-network cumulant growth. The result is a criterion for when low-rank networks remain trainable for structural reasons, not merely parameter-efficient ones.

Neural networks often struggle to learn highly oscillatory functions at finite training time: low-frequency components are fitted first, while high-frequency modes can remain poorly recovered even when the model is expressive enough to represent them. Low-rank networks are usually introduced as compressed alternatives to dense models, but this view overlooks a more useful possibility: rank can act as a structural control on what the network learns first. In this paper, we show that this is the case. We first prove that low-rank random-feature networks in the mean-field limit converge to a global minimizer of the population risk whenever their limiting dynamics converge. We then show that the rank is not merely a compression parameter: choosing it correctly can reduce the number of trainable degrees of freedom while also improving the fit of highly oscillatory targets. The key practical message is that the best rank is typically intermediate. If the rank is too small, the model lacks expressivity; if it is too large, it recovers the finite-time bias of the dense model. Controlled geometric and Fourier diagnostics, together with high-frequency regression experiments, show that an appropriate low rank can lower test loss, improve high-frequency recovery, and that the optimal rank shifts with the target spectrum and training objective.

In this communication, we aim to present the solving of an MBDA’s use case related to optimal assignment, onto IBM online QPUs. The Quantum Approximate Optimization Algorithm (QAOA) (Farhi et al. 2014) is the base of our Variational Quantum Algorithm developed. We compare two methods to account for constraints, first primarily by integrating them into the Cost Hamiltonian with Lagrangian multipliers and second, by adapting the Mixer Hamiltonian according to (Wang et al. 2022) and (Fuchs et al. 2022). For the former, determining the optimal Lagrangian multipliers is generally a challenging task and the integration of constraints into the Cost Hamiltonian can significantly increase the associated circuit depth. The latter method aims to reduce the overall Hilbert space to only feasible solutions, which lets get rid of Lagrangian multipliers but may significantly enlarge the circuit associated to the Mixer Hamiltonian and make the initial state harder. It is then interesting to compare the circuit depth of both methods with respect to how well they are able to statistically put forward optimal solutions against non-optimal and non-feasible ones, still for relatively small sized instances, to fit on current QPUs.