Diffusion-Augmented LLM

A diffusion-augmented LLM is a hybrid architecture introduced by Subham Sekhar Sahoo and coauthors in "Unlocking Lossless Speedups in LLMs via Discrete Diffusion" (arXiv:2609.04010, September 2026). It splits a language model's parameters into two sets. Autoregressive (AR) weights, trained with the standard next-token-prediction objective, define the model's output distribution and quality. A separate, lightweight diffusion adapter is trained afterward through a short "Diffusion Distillation" phase that adds negligible overhead to normal LLM training. This adapter learns to draft several tokens at once from that same distribution.

At inference, the diffusion adapter proposes multiple tokens in parallel. A sampler the authors call Psi-Spec accepts or corrects them so the final output is provably identical in distribution to what the AR model alone would have produced. This lossless guarantee is similar in spirit to speculative decoding's accept-reject scheme. Unlike speculative decoding, the approach needs no separately trained draft model, since the same base model supplies both components. Unlike standalone diffusion language models, it does not trade away the AR model's output quality to gain parallelism.

The paper names the resulting model family Uno and reports throughput gains of up to 3x over the base autoregressive model, outperforming speculative-decoding baselines at every tested batch size. An 8B Uno model reportedly outperformed the 26B open diffusion model DiffusionGemma and the proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks. Diffusion adapters can also be added to existing open-weight AR models at under 4% of their parameter count, rather than trained only from scratch.