Q Labs Introduces Dust, a Backpropagation-Free Method for Transformer Pretraining

Q Labs has released Dust, a zeroth-order method that pretrains transformer language models without the conventional backward pass, using random activation perturbations and token-level rewards to estimate weight updates. The October 2026 research note reports that Dust can approach, and in some smaller-token settings beat, tuned backpropagation baselines, but the authors say it is not yet compute-efficient enough to replace backpropagation in practical training.