# Neural Networks: Explained

#1 May 13, 2024 Foundation

https://www.aibutsimple.com/p/neural-networks-explained

In the past decade, the best AI systems (such as image generators, or a machine translation service), have been using a technique called deep learning.

- Deep learning is an approach to AI using neural networks. So how did neural networks get so popular?

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/4739b7a0-999f-4275-8dbb-538ef43a72af/Add_a_heading__4_.png)

Here’s a simple timeline of neural networks:

- Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts, two University of Chicago researchers.
- Neural networks were a major area of research in both neuroscience and computer science but have had a rough career: dying then resurging, then dying again, but finally resurging fueled by the increased processing power of graphics cards.

Neural networks are a very versatile tool. They are a means of learning how to perform a task by analyzing training examples.

Let’s say you have a function. If you are given this function explicitly, then it is 100% guaranteed to replicate the correct output, given any input.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/32040769-bcf0-4605-9467-725b1eb72fd8/Add_a_heading__1_.png)

But what if you weren’t given the function at all, and instead, some data points and wanted to guess what the function was shaped like?

For instance, there is no function that can tell you whether a tumor is malignant or benign. But using neural networks, we can give it some data and tell it to make some sense of it.

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0c0445d6-b7f7-42e2-905f-665157e4f486/Add_a_heading__2_.png)

If we train it well, then it can predict whether a given tumor is malignant or benign with decent precision.

The data fed into a neural network will be different for different use cases. For instance, in an object recognition system, the network might be fed thousands of labeled images of cars, houses, and other objects.

Neural networks are extremely interesting, since they are modeled after a human brain.

It consists of many neurons that are densely interconnected, organized in layers:

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/c2e52240-f40a-4999-b8df-7dcd09530eac/Copy_of_Add_a_heading__1_.png)

Many of today’s neural networks are feed-forward, meaning that they pass data only forward.

To each of its incoming connections, a node will assign a number known as a “weight.”

When the network is active, the node receives a different input (just a number) over each of its connections and multiplies it by the associated weight.

In this process, you add an additional bias value, that can be tweaked during training (will go into depth in next weeks’ letter!).

It then adds the resulting products together, yielding a single number.

The number is put through an activation function, which controls the influence of that node’s data on the rest of the network.

- It is used to add some non-linearity into the network, since always letting the data feed forward would result in a linear approximation.
- As an example, if you were to use a “ReLu” activation function, It would allow you to only pass data from certain neurons in a network (see example below)

![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,width=1200,format=auto,onerror=redirect,quality=80/uploads/asset/file/0e31c74d-1cdc-4130-be5b-53b49fcb6d2d/Add_a_heading__3_.png)

The node then sends the data — the sum of the weighted inputs — along its outgoing connections.

When a neural net is being trained, all of its weights and thresholds are initially set to random values.

- Nowadays, researchers are testing certain initial values to optimize the learning process.

Training data is fed to the first layer, the input layer, and it passes through the succeeding layers, getting multiplied and added together in complex ways, until it finally arrives, at the output layer.

During training, the weights and biases are continually adjusted until a desired result.

- This usually means at a low enough loss to provide consistent results for a certain use case.
  - The loss is simply a measure of how good or bad a neural network performs
  - It’s how much the network’s output differs from the correct one

And that’s a simple overlook of a neural network.

Of course, there will be more complicated concepts that can be added on, but for now, this is generally how a neural network works.

---

If you enjoy our content, consider supporting us so we can keep doing what we do. Please share this with a friend!

If you like, you can also donate to our team to push out better newsletters every week!

[Support us ☕](https://ko-fi.com/aibutsimple?utm_source=newsletter.aibutsimple.com&utm_medium=referral&utm_campaign=neural-networks-explained)

---

That’s it for the first issue of AI, but simple. See you next week!

—AI, but simple team