Introduction to Deep Learning

Deep learning is a subset of machine learning using neural networks, allowing a machine to learn complicated patterns and relationships within data.

It is a pivotal technology behind innovations like Large Language Models (LLMs), self-driving cars, Natural Language Processing (NLP), and object detection.

Deep learning focuses on Artificial Neural Networks (ANNs) with many layers, often referred to as deep neural networks.

These networks are capable of learning complex patterns from data by stacking multiple layers (hence the term “deep”) with many neurons.

Here’s a simple timeline of the history of neural networks:

• Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts, two researchers from the University of Chicago.

• Neural networks were a major area of research in both neuroscience and computer science, but it has had a rough career: dying, then resurging, then dying again, but finally resurging, fueled by the increased processing power of graphics cards.

Neural networks do not require explicit programming to handle specific conditions but instead rely on training to learn difficult patterns.

This makes them ideal for solving a wide range of problems in the real world.

Neural networks are function approximators. They approximate a function based on the input data and true outputs (labels) you give it, using an optimization algorithm to learn based on how close the neural network’s output is compared to the true output.

During training, the neural network learns the function that transforms the input data into the true outputs.

To put this into perspective, we’ll use an example. Let’s say you have a function. If you are given this function explicitly, then it is 100% guaranteed to replicate the correct output, given any input.

But what if you weren’t given the function at all and instead, some data points and wanted to guess what the function was shaped like?

For instance, there is no function that can tell you whether a tumor is malignant or benign. But using neural networks, we can give it some data and tell it to make some sense of it.

If we train it well, then it can predict whether a given tumor is malignant or benign with decent precision, good enough for real-world applications (while not being 100% accurate).

In math terms, the neural network learns a function f_hat​, parameterized by weights (W) and biases (b), such that: