ResNet (Residual Network)

ResNet, short for Residual Network, is a convolutional neural network architecture designed to make training very deep networks practical and effective. Introduced by Kaiming He and colleagues at Microsoft Research in 2015, it addressed a problem in deep learning: as networks grow deeper, gradients tend to vanish or explode during backpropagation, causing training to stall and performance to degrade. ResNet's solution was to have each stack of layers learn a residual function, the difference between the desired output and the input itself, rather than a direct mapping from input to output.

The mechanism enabling this is the skip connection (also called a shortcut connection), which routes the input of a block directly to its output, bypassing one or more intermediate layers. The result is added element-wise to the transformed output before passing to the next block. Mathematically, if a block learns a function F(x), the block's output becomes F(x) + x rather than F(x) alone. This change allows gradients to flow backward through the skip path without being attenuated by layer after layer of transformations, making it feasible to train networks with hundreds or even thousands of layers.

ResNet's impact was visible when it won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2015, achieving top-5 error rates that surpassed all prior architectures by a significant margin. The original paper introduced variants ranging from ResNet-18 to ResNet-152, with the number indicating total layer depth. Subsequent work extended the concept further, producing architectures like ResNeXt, Wide ResNet, and DenseNet, all of which build on the residual learning principle.

Beyond image classification, ResNet has become a common backbone across computer vision tasks including object detection, semantic segmentation, and medical image analysis. Its design principles have also influenced architectures outside vision, including components of modern natural language processing models. The residual connection concept is now used as a standard building block in deep learning.