---
title: Spacetime Patches
type: vocabulary
url: "https://www.envisioning.com/vocab/spacetime-patches"
summary: A method for tokenizing video into spatial and temporal chunks for transformer-based models.
year: 2024
generality: 0.10
---

# Spacetime Patches

A method for tokenizing video into spatial and temporal chunks for transformer-based models.
Spacetime patches are a video tokenization technique that divides raw video data into discrete three-dimensional chunks, each capturing a small region of space across a short span of time. Rather than treating video as a flat sequence of frames or a single high-dimensional tensor, this approach segments the data along both spatial axes (height and width) and the temporal axis (time), producing compact units that encode local motion and appearance together. These patches can then be flattened into sequences and fed into transformer architectures, much like image patches are used in Vision Transformers (ViTs).

The practical value of spacetime patches lies in their flexibility. Because each patch is a self-contained unit of spatiotemporal information, models trained on them can handle videos of varying resolutions, aspect ratios, and durations without requiring normalization steps like resizing or padding. A model simply processes however many patches the input produces, making the approach natively resolution-agnostic. This is a meaningful departure from earlier video models that required fixed input dimensions and struggled to generalize across different video formats.

Spacetime patches gained widespread attention with OpenAI's Sora, a large-scale diffusion transformer model capable of generating high-fidelity video from text prompts. Sora compresses raw video into a lower-dimensional latent space and then applies spacetime patch tokenization within that space, allowing the model to operate efficiently even on long, high-resolution clips. The technique enables Sora to train on heterogeneous video datasets without enforcing a uniform format, which is credited as a key factor in its generalization ability.

More broadly, spacetime patches represent a convergence of ideas from image tokenization, video understanding, and scalable transformer design. As video generation and understanding become increasingly central to AI research, this tokenization strategy is likely to influence how future multimodal models ingest and reason about dynamic visual content. It reflects a broader trend of adapting the patch-based paradigm—proven effective in image models—to the richer, higher-dimensional structure of video.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/spacetime-patches)