Issue #14 Architecture Edwin Dong
Modern Convolutional Neural Network Architectures
In previous issues, we’ve explored the many different types of neural networks, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers, among others.
However, we’ve only scratched the surface of what’s possible with these types of networks, and in recent years, they have gotten extremely powerful.
In this issue, we’re going to focus on one of the building blocks of deep learning, an architecture that accelerated the massive growth of neural networks: the Convolutional Neural Network (CNN).
A standard CNN design.
If you don’t know anything about these networks, check out our issue on CNNs to learn the basics before continuing (padding, stride, conv window size, etc.)
Although Convolutional Neural Networks haven’t seen as much work in the past 5 years, they used to be a popular research topic during the 2010s.
As computer vision and image processing became more and more popular during that time, CNNs rose to fame.
Before the 2010s, CNNs existed, but they weren’t considered to be effective or efficient for computer vision and difficult real world image processing tasks.
It was only in 2012 when CNNs started to take off and become a crucial tool in many deep learning engineers’ toolkits.
It was one groundbreaking architecture that proved neural networks learning the features could perform better than manually selected features in computer vision.
This architecture is known as AlexNet, which is an 8 layer CNN that won the ImageNet Large Scale Visual Recognition Challenge 2012 by a large margin, accelerating the adoption of neural networks for image tasks.
