Circuit

In AI and machine learning, a circuit refers to an organized collection of processing elements—gates, neurons, attention heads, or other computational primitives—and their interconnections that together implement a specific function or transformation. The concept spans multiple levels of abstraction: physical electronic logic circuits, formal Boolean and circuit-complexity models used in theoretical computer science, and the emergent functional subnetworks discovered inside trained neural networks. Across all these uses, the circuit framing provides a compact way to reason about computational capacity, depth, parallelism, and compositionality.

In theoretical ML, circuit complexity offers formal tools for bounding what classes of functions neural networks can represent given constraints on depth, width, and gate type. Boolean circuits and arithmetic circuits serve as idealized models that connect neural network expressivity to classical complexity-theoretic results, helping researchers understand which problems are tractable under which architectural constraints. This theoretical lens informs choices about inductive biases, activation functions, and architectural depth.

The most active contemporary use of the circuit concept in ML comes from mechanistic interpretability, where researchers treat pathways of activations and weights inside trained models as algorithmic modules that can be identified, probed, and causally manipulated. By tracing how information flows through specific layers and components, investigators can isolate subnetworks responsible for particular behaviors—such as indirect object identification in language models or curve detection in vision networks. Techniques like activation patching, ablation studies, and causal mediation analysis operationalize this circuit-level view, enabling researchers to attribute model behaviors to specific computational structures rather than treating the network as an opaque black box.

Circuit-level analysis has significant practical implications: it enables targeted model editing, diagnosis of failure modes and biases, more principled distillation, and the design of modular architectures. It also bridges empirical deep learning with classical theory, grounding observations about learned representations in a rigorous vocabulary inherited from logic, complexity theory, and dynamical systems. As models grow larger and more capable, the circuit framework has become one of the primary conceptual tools for understanding what computations neural networks are actually performing.