AI-generated review 5 min read Beginner 878 words

VGG: Very Deep Convolutional Networks

VGG showed that stacking small 3x3 convolutions into 16-19 layer CNNs wins at ImageNet classification — the simple deep-learning blueprint behind modern vision.

vgg cnn computer-vision image-classification deep-learning
Computer vision · 2014

Primary source: Very Deep Convolutional Networks for Large-Scale Image Recognition (arXiv:1409.1556)

When AlexNet won ImageNet in 2012 with 8 learned layers, it left an obvious question hanging: what happens if you simply go deeper? The 2014 paper Very Deep Convolutional Networks for Large-Scale Image Recognition, by Karen Simonyan and Andrew Zisserman of Oxford's Visual Geometry Group, answered that question with unusual thoroughness. Instead of proposing exotic new modules, the authors ran a controlled experiment on depth itself, pushing a plain convolutional network to 16 and 19 layers — and nearly winning the ILSVRC 2014 classification challenge with it.

Why network depth matters for image classification

A convolutional network builds its understanding of an image in a hierarchy. Early layers respond to edges and color blobs, middle layers to textures and parts, and late layers to whole objects. Visualizations of these features, such as those in Zeiler and Fergus, show this progression clearly: depth buys abstraction.

But before VGG, nobody knew how far this scaled. Deeper networks are more expressive, yet harder to train — gradients fade, and bigger models overfit. VGG's contribution was to isolate depth as the single variable: fix everything else about the architecture, vary only the number of layers from 11 to 19, and measure what happens on ImageNet's 1,000-class challenge.

The 3x3 convolution stack: small filters, deep networks

Single 7x7 filter: 49 weights per channel 7 x 7 one ReLU vs Stack of three 3x3 filters: 27 weights per channel 3x3 3x3 3x3 ReLU ReLU ReLU
Same 7x7 receptive field, two designs: one large filter versus three stacked 3x3 filters with fewer parameters and three non-linearities instead of one.

VGG's central design decision was to use only tiny 3 \times 3 convolution filters with stride 1 throughout the network. A stack of two 3 \times 3 layers has the same effective receptive field as a single 5 \times 5 layer, and three stacked 3 \times 3 layers cover the same 7 \times 7 region as one large filter. The stacked version wins twice: it needs far fewer parameters (3 \cdot 3^2 = 27 versus 7^2 = 49 weights per channel pair), and it places a ReLU non-linearity after each layer, making the decision function more discriminative. Spatial resolution is reduced only by five 2 \times 2 max-pooling layers, and a 1 \times 1 convolution appears in one configuration as a linear projection followed by a ReLU.

VGG-16 and VGG-19: configurations and training

The paper evaluates six configurations of increasing depth, but two became famous. VGG-16 (configuration D) has 13 convolutional layers plus 3 fully connected layers, and VGG-19 (configuration E) has 16 convolutional layers plus 3 fully connected layers. Both take 224 \times 224 RGB images, pass them through blocks of 3 \times 3 convolutions with 64, 128, 256, 512 and 512 channels separated by max-pooling, and end in three fully connected layers of size 4096, 4096 and 1000 with a softmax. VGG-16 holds around 138 million parameters, most of them in the dense layers. Notably, the network drops the local response normalization used by AlexNet — with deep stacks and small filters, it adds cost without helping — and regularizes the fully connected layers with dropout at rate 0.5.

Training 19 layers without batch normalization or residual connections took care. The authors pre-trained an 11-layer network first, then used its weights to initialize the first four convolutional layers and the last three fully connected layers of the deeper nets, initializing the middle layers randomly. They trained on multi-scale inputs: rather than fixing the image size, they jittered the scale across a wide range, which acts as extra data augmentation and lets a single model handle objects at different sizes. Optimization was mini-batch gradient descent with momentum 0.9 and weight decay, a standard recipe that the sheer depth made newly demanding.

What VGG proved on ImageNet

The results settled the depth question decisively. On ILSVRC 2014, VGG took second place in classification behind GoogLeNet and first place in the localization track. Its ensemble reached roughly 7.3% top-5 error, and accuracy improved steadily as layers were added from 11 to 19. Just as importantly, the paper showed that features learned on ImageNet transfer: the same deep representations achieved strong results on other datasets and tasks without redesign. Depth was not a quirk of one benchmark — it was a general principle, and within a year ResNet would push it past 100 layers using VGG's 3 \times 3 stack as its basic building block.

Why VGG became everyone's backbone

VGG's plain, uniform structure — repeat 3 \times 3 convolutions, pool, repeat — made it the easiest modern network to understand, modify and borrow from. Its convolutional blocks became the default feature extractor inside detectors, segmentation models such as U-Net-style encoders, and the neural style transfer work that followed. Even today, when vision transformers dominate leaderboards, VGG remains the standard teaching architecture: everything it does is visible, with no branching or skip connections to trace. Its lesson endures in every deep network since — stack small, simple layers, and let depth do the work.

Bottom line

VGG proved that depth itself is the engine of image recognition: stacking small 3 \times 3 filters to 16–19 layers nearly won ILSVRC 2014 and produced features that transferred everywhere. Simple, deep, and uniform beat clever and shallow.

Source

  • Simonyan & Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, 2014. arXiv:1409.1556
  • Szegedy et al., Going Deeper with Convolutions, 2014. arXiv:1409.4842
  • He et al., Deep Residual Learning for Image Recognition, 2015. arXiv:1512.03385

Want the next review in your inbox? The briefing collects them — at most one email a week. Read signup data and removal options.

Cite this report
@misc{knowyourml-vgg-very-deep-convolutional-networks
  title        = {VGG: Very Deep Convolutional Networks},
  author       = {{Know Your ML}},
  year         = {2014},
  howpublished = {\url{https://knowyourml.yunusemrevurgun.com/report.php?slug=vgg-very-deep-convolutional-networks}},
  note         = {AI-generated review of arXiv:1409.1556. Primary sources are listed on the page.}
}
Machine Learning Decision Toolkit — decision matrix, scorecards and trade-off analysis
Toolkit 7 decision tools Model-agnostic

Machine Learning Decision Toolkit

Turn this review into a decision: the Machine Learning Decision Toolkit packages the evaluation scorecards, workload templates and deployment checks behind model, provider and architecture choices.

$24 View on Gumroad

References

Frequently asked questions

What does VGG stand for?

VGG stands for the Visual Geometry Group at the University of Oxford, the lab of Simonyan and Zisserman. The name stuck because the architecture became one of the most reused designs in computer vision.

Why does VGG use only 3x3 convolutions?

A stack of two 3x3 convolutional layers has the same effective receptive field as one 5x5 layer, and three 3x3 layers match one 7x7 layer. The stacked version uses fewer parameters and adds more non-linearities, which makes the network more expressive.

What is the difference between VGG-16 and VGG-19?

VGG-16 has 16 weight layers (13 convolutional plus 3 fully connected) and VGG-19 has 19. VGG-19 is marginally more accurate but slower and larger, so VGG-16 is the more commonly used variant.

Is VGG still used today?

Modern architectures such as ResNet are more accurate and far more parameter-efficient, so VGG is rarely trained from scratch for new projects. It remains popular for teaching, transfer learning baselines, and tasks like style transfer.

How to Remain Valuable When Intelligence Becomes Cheap — book cover
eBook 224 pages

How to Remain Valuable When Intelligence Becomes Cheap

This report covered one more way the machine gets smarter. The companion eBook asks the question on the other side: which human advantages stay scarce — and valuable — when intelligence itself is cheap?

Read the eBook