ResNet: Deep Residual Learning for Image Recognition
An AI-generated review of ResNet, the 2015 architecture that used skip connections to train networks hundreds of layers deep and became the default backbone.
Primary source: Deep Residual Learning for Image Recognition (arXiv:1512.03385)
By 2015, the formula "deeper is better" had driven vision networks from 8 layers (AlexNet) to 16–19 (VGG) to 22 (GoogLeNet). Then progress stalled in a puzzling way: networks with more layers started underperforming shallower ones on training error, not just validation error. Kaiming He and colleagues at Microsoft Research diagnosed the problem and solved it with a deceptively simple change — skip connections — creating ResNet, the architecture that won ImageNet 2015 and became the most reused backbone in deep learning.
The core idea
Deeper networks should at least be able to copy a shallower network's function by learning identity mappings on the extra layers. The fact that they could not — training error increased with depth — suggested an optimization problem rather than a capacity problem: gradients vanished or became unstable as they flowed through many stacked nonlinearities.
The fix: instead of forcing stacked layers to learn the desired mapping directly, let them learn the residual — the difference between the desired mapping and the input. A residual block computes:
output = F(x) + x
where F is a few weight layers and x is the block input, added via a skip connection. If the identity mapping is optimal, the layers only need to learn to output zero, which is far easier for an optimizer than learning an identity transform through nonlinearities.
How it works
A ResNet stacks residual blocks, each with two or three convolutional layers (batch normalization between them), with skip connections bypassing the block. To change feature-map dimensions, the skip connection uses a 1×1 convolution with stride 2 rather than an identity shortcut.
The paper evaluated 34-, 50-, 101-, and 152-layer variants. Key results:
- The 34-layer ResNet beat the 34-layer plain network on both training and validation error, confirming the optimization story.
- A 152-layer ResNet achieved a 3.57% top-5 error on ImageNet, winning ILSVRC 2015 — better than human-level performance (about 5.1%) on that benchmark.
- Deep variants (50+) use "bottleneck" blocks — 1×1, 3×3, 1×1 convolutions — that keep FLOPs and parameters comparable to shallower VGG nets while being much deeper.
Why it matters
- Depth became trainable. ResNet showed that hundreds of layers could be optimized reliably, and modern transformers, diffusion U-Nets, and dense prediction heads all inherit the residual design.
- Default backbone. ResNet-50/101 remain the standard feature extractor for detection (Faster R-CNN), segmentation (DeepLab), and countless transfer-learning pipelines.
- Conceptual bridge. The residual idea generalizes beyond vision: highway networks, transformer residual sublayers, and modern optimizer research all build on the insight that gradients need stable, near-identity paths.
Limitations and open questions
Residual connections reduce the vanishing-gradient problem but do not eliminate it; very deep ResNets still need careful initialization and normalization. The paper's ablations focused on image classification; the "why" of residual learning — whether it smooths the loss landscape, shortens gradient paths, or both — was debated for years after. Later work (DenseNet, ResNeXt, EfficientNet) showed the residual principle could be pushed further in different directions, and vision transformers later challenged the necessity of convolutions entirely.
Key takeaways
- ResNet's core insight: learn residuals with skip connections instead of full mappings.
- Skip connections fix the optimization problem of depth, not just the accuracy problem.
- The 152-layer model won ILSVRC 2015 with 3.57% top-5 error, beating human-level performance on ImageNet.
- Residual blocks are now a universal building block across CNNs, transformers, and diffusion models.
Want the next review in your inbox? The briefing collects them — at most one email a week. Read signup data and removal options.
Cite this report
@misc{knowyourml-resnet-deep-residual-learning
title = {ResNet: Deep Residual Learning for Image Recognition},
author = {{Know Your ML}},
year = {2015},
howpublished = {\url{https://knowyourml.yunusemrevurgun.com/report.php?slug=resnet-deep-residual-learning}},
note = {AI-generated review of arXiv:1512.03385. Primary sources are listed on the page.}
}
Machine Learning Decision Toolkit
Turn this review into a decision: the Machine Learning Decision Toolkit packages the evaluation scorecards, workload templates and deployment checks behind model, provider and architecture choices.
References
- Deep Residual Learning for Image Recognition (arXiv:1512.03385) — https://arxiv.org/abs/1512.03385
Part of
Beyond the paper
How to Remain Valuable When Intelligence Becomes Cheap
This report covered one more way the machine gets smarter. The companion eBook asks the question on the other side: which human advantages stay scarce — and valuable — when intelligence itself is cheap?