AlexNet
Landmark deep convolutional network that ignited the modern deep learning revolution in 2012.
AlexNet is a deep convolutional neural network architecture that achieved a breakthrough performance on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, reducing the top-5 error rate to 15.3%, nearly 11 percentage points better than the runner-up. Developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto, the network showed that deep learning could dramatically outperform hand-engineered feature extraction methods on large-scale visual recognition tasks, helping to launch the modern deep learning era.
The architecture consists of five convolutional layers followed by three fully connected layers, processing 224×224 RGB images into 1,000 class probability scores. Several design choices were novel at the time. The use of Rectified Linear Unit (ReLU) activations instead of sigmoid or tanh functions sped up training. Overlapping max-pooling reduced spatial dimensions while preserving salient features. Local response normalization provided a form of lateral inhibition inspired by neuroscience. The entire network was trained on two NVIDIA GTX 580 GPUs in parallel, an early demonstration that commodity graphics hardware could make large-scale deep learning tractable.
AlexNet also popularized dropout as a regularization technique, randomly deactivating neurons during training to prevent co-adaptation and reduce overfitting on the 1.2-million-image dataset. Data augmentation through random cropping, flipping, and color jittering further improved generalization. These techniques formed a practical recipe that subsequent architectures, including VGGNet, GoogLeNet, and ResNet, would refine and build upon.
AlexNet's significance goes beyond its benchmark results. It shifted the research community's attention toward end-to-end learned representations, spurred massive investment in GPU computing infrastructure, and established ImageNet competition performance as the standard benchmark for computer vision progress for nearly a decade. The 2012 paper, "ImageNet Classification with Deep Convolutional Neural Networks," remains one of the most cited works in the history of machine learning.
Sources
- ImageNet Classification with Deep Convolutional Neural Networks
NeurIPS · Jan 1, 2012
Research this in Signals
Signals turns a topic into a sourced research record you can inspect and rerun. Your first scan is free, and this one starts with AlexNet already loaded, so edit it or scan as is.
