A neural network is not magic.

It is a system that transforms input into output through layers.

The real question is:

How does that system learn from mistakes?

Core Idea

A neural network takes data, passes it through connected layers, and produces an output.

During training, it adjusts internal values so future outputs become better.

Those internal values are mainly weights and biases.

So the simplest view is:

Input → Layers → Output → Error → Update

That loop is the foundation of neural network learning.

The Key Structure

A basic neural network looks like this:

Input Layer → Hidden Layers → Output Layer

Each layer transforms the data.

A neuron usually computes:

z = w · x + b

Then an activation function transforms it:

a = activation(z)

Where:

• x = input
• w = weight
• b = bias
• z = raw score
• a = activated output

This is the basic building block.

Implementation View

At a high level, training works like this:

take input data

run forward propagation

compute prediction

compare prediction with target

compute loss

run backpropagation

update weights and biases

repeat

This is why neural networks are trainable.

They do not just compute outputs.

They use errors to adjust themselves.

Concrete Example

Imagine a small model that predicts whether an email is spam.

The input features might include:

• suspicious words
• sender reputation
• number of links
• message length

The network processes those inputs.

It predicts:

spam probability = 0.82

If the real label is spam, the prediction is good.

If the real label is not spam, the network needs to adjust.

Backpropagation tells the model which weights contributed to the mistake.

Then training updates those weights.

Perceptron vs MLP

The perceptron is the simplest starting point.

It takes inputs, applies weights, adds a bias, and produces an output.

But a single perceptron is limited.

It can only represent simple decision boundaries.

A Multi-Layer Perceptron expands this idea.

It stacks multiple layers.

Perceptron:

• one basic computational unit
• simple input-output mapping
• limited representational power

MLP:

• multiple connected layers
• hidden representations
• more expressive decision boundaries

So the MLP is where neural networks become more useful.

Forward Propagation vs Backpropagation

This comparison is the core of learning.

Forward propagation computes the prediction.

Backpropagation computes how to improve it.

Forward propagation:

• moves from input to output
• calculates activations
• produces prediction
• computes loss

Backpropagation:

• moves from loss backward
• computes gradients
• assigns error responsibility
• prepares parameter updates

Forward pass answers:

“What did the model predict?”

Backward pass answers:

“What should change?”

Why Activation Functions Matter

Stacking linear layers is not enough.

Without activation functions, multiple layers can collapse into one linear transformation.

That means the model cannot represent complex patterns well.

Activation functions add nonlinearity.

This lets neural networks learn curves, boundaries, and deeper representations.

In short:

Linear layers calculate.

Activation functions reshape.

Both are needed.

What Actually Changes During Learning?

When a neural network learns, it updates parameters.

The most important parameters are:

• weights
• biases

Weights control how strongly inputs affect the output.

Biases shift the output.

A simple update idea is:

new parameter = old parameter - learning rate × gradient

The gradient tells the direction.

The learning rate controls the step size.

That is the practical meaning of learning.

From Neural Networks to Deep Learning

A neural network becomes “deep” when it has many layers.

But depth alone is not enough.

Deep learning depends on:

• layered representations
• nonlinear activations
• backpropagation
• parameter updates
• enough data and compute

The deeper the network, the more abstract its internal representations can become.

For example:

In image models:

• early layers may detect edges
• middle layers may detect shapes
• deeper layers may detect objects

That is why neural networks became the foundation of modern deep learning.

Recommended Learning Order

If neural networks feel scattered, learn them in this order:

1. Perceptron
2. Multi-Layer Perceptron
3. Neural Network
4. Forward Propagation
5. Backpropagation
6. Activation Function
7. Weights and Biases
8. Model Parameters
9. Deep Learning

This order works because you first understand the basic unit.

Then you understand the network.

Then you understand computation, learning, and deep extensions.

Takeaway

A neural network is a trainable system of layered transformations.

The shortest version is:

Neural Network = layers + activations + parameters + learning

Forward propagation makes predictions.

Backpropagation computes corrections.

Weights and biases store what the model has learned.

If you remember one idea, remember this:

Neural networks learn by repeatedly predicting, measuring error, and updating internal parameters.

Discussion

When learning neural networks, do you find it easier to start from the perceptron, or from the full forward/backpropagation training loop?

Originally published at zeromathai.com.
Original article: https://zeromathai.com/en/neural-network-hub-en/

GitHub Resources
AI diagrams, study notes, and visual guides:
https://github.com/zeromathai/zeromathai-ai