CUDA (Compute Unified Device Architecture)

CUDA is a parallel computing platform and programming model developed by NVIDIA that lets developers use the parallelism of graphics processing units (GPUs) for general-purpose computation. Introduced in 2007, it provides extensions to standard languages like C, C++, and Fortran that let programmers write code targeting the GPU's many specialized processing units. A CPU optimizes for low-latency sequential execution across a handful of cores, while a GPU runs thousands of lightweight threads simultaneously, which suits the dense matrix and tensor arithmetic used in modern machine learning.

CUDA exposes a hierarchy of parallelism through threads, thread blocks, and grids, which map onto the GPU's physical streaming multiprocessors. Developers write kernels, functions that execute in parallel across many threads, and manage data movement between CPU memory (host) and GPU memory (device). NVIDIA's ecosystem around CUDA includes libraries such as cuBLAS for linear algebra and cuDNN for deep neural network primitives, which deep learning frameworks like PyTorch and TensorFlow use internally. Because of this layered ecosystem, most practitioners use CUDA without writing low-level GPU code.

CUDA's role in AI has been substantial. Training deep neural networks orders of magnitude faster than on CPUs helped enable the deep learning advances of the early 2010s. AlexNet's 2012 ImageNet victory was made practical largely because CUDA let researchers train large convolutional networks in days rather than months. Today, most deep learning training and inference pipelines run on CUDA-enabled hardware, and GPU availability is a primary constraint in large-scale AI development.

CUDA is also used in scientific simulation, computational finance, medical imaging, and other domains requiring high-throughput numerical computation. Alternative platforms like OpenCL and AMD's ROCm exist, but CUDA's maturity, library support, and tight hardware-software integration have made it the de facto standard for GPU-accelerated AI research and production.