DAG (Directed Acyclic Graph)

A Directed Acyclic Graph (DAG) is a graph structure composed of nodes (vertices) connected by directed edges, where the directionality enforces a one-way relationship between nodes and the acyclic constraint ensures no path can loop back to its starting point. This combination makes DAGs ideal for representing hierarchical dependencies, ordered sequences, and causal relationships — any domain where some elements must logically precede others. In mathematics and computer science, DAGs underpin topological sorting, scheduling algorithms, and version control systems, but their role in machine learning has made them especially prominent in modern AI research and engineering.

In machine learning, DAGs appear in several critical contexts. Probabilistic graphical models such as Bayesian networks use DAGs to encode conditional independence relationships among random variables, where each node represents a variable and each directed edge encodes a probabilistic dependency. This structure allows efficient inference and reasoning under uncertainty. DAGs also formalize causal models in causal inference, where directed edges represent cause-and-effect relationships rather than mere correlations, enabling researchers to reason about interventions and counterfactuals in a principled way.

Perhaps the most pervasive use of DAGs in modern AI is in deep learning computation graphs. Frameworks like TensorFlow and PyTorch represent neural network computations as DAGs, where nodes correspond to mathematical operations and edges represent the flow of tensors between them. The acyclic structure guarantees a well-defined forward pass for computing predictions and a well-defined backward pass for computing gradients via backpropagation. This graph-based representation also enables powerful optimizations such as automatic differentiation, operation fusion, and parallel execution. More recently, DAG-structured architectures have been explored directly as neural network topologies, as in neural architecture search methods that treat the network design space as a DAG over possible operations.