Video DeltaNet

Video DeltaNet (VDN) is a hybrid attention architecture for video diffusion transformers, introduced in "Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation" (arXiv:2609.20744, submitted 17 September 2026) by Haocheng Xi and ten coauthors. It pairs local softmax attention for short-range frame-to-frame interactions with a bidirectional linear-memory branch that carries long-range context across a clip. Separate output projections and learnable gates calibrate how much each branch contributes at a given layer.

The linear branch centers on Video Delta Attention (VDA), which updates its memory once per frame by folding in that frame's spatial tokens jointly, rather than token by token. VDA extends DeltaNet, a linear-attention mechanism built for sequence models, to the spatiotemporal token grids that video diffusion transformers process. A staged teacher-alignment distillation recipe introduces this pathway into an already pretrained video model gradually, avoiding a full retrain.

The authors instantiate VDN on MiniMax H3 to produce VDN-H3, applying the hybrid attention to video-to-video interactions while keeping full softmax attention for text or audio interactions. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 denoises a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs. That is a 14.5x speedup over the 50-step dense H3 baseline on the same hardware. Code and weights are published on GitHub and Hugging Face under the OpenVDN account.