{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "Daniel Warfield",
  "title_listed": "Neural Networks - Intuitively and Exhaustively Explained",
  "keywords": "神经网络,直观,详解",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://towardsdatascience.com/neural-networks-intuitively-and-exhaustively-explained-0153f85c1007/",
  "publish_date": "2023-05",
  "corpus_stage": "预训练/SFT",
  "notes": "神经网络直观且详尽讲解",
  "id": "5361b27a4c83",
  "collected_at": "2026-10-01T21:47:52",
  "title": "Neural Networks - Intuitively and Exhaustively Explained | Towards Data Science",
  "body_markdown": "Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Neural Networks - Intuitively and Exhaustively Explained\nAn in-depth exploration of the most fundamental architecture in modern AI.\n[Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\nFebruary 3, 202525 min read\n### An in-depth exploration of the most fundamental architecture in modern AI\n!["The Thinking Part" by Daniel Warfield using MidJourney. All images by the author unless otherwise specified. Article originally made available on Intuitively and Exhaustively Explained.](../images/000.png)\n\"The Thinking Part\" by Daniel Warfield using MidJourney. All images by the author unless otherwise specified. Article originally made available on [Intuitively and Exhaustively Explained](https://iaee.substack.com/).\nIn this article we'll form a thorough understanding of the neural network, a cornerstone technology underpinning virtually all cutting edge AI systems. We'll first explore neurons in the human brain, and then explore how they formed the fundamental inspiration for neural networks in AI. We'll then explore back-propagation, the algorithm used to train neural networks to do cool stuff. Finally, after forging a thorough conceptual understanding, we'll implement a Neural Network ourselves from scratch and train it to solve a toy problem.\n* * *\n**Who is this useful for?** Anyone who wants to form a complete understanding of the state of the art of AI.\n**How advanced is this post?** This article is designed to be accessible to beginners, and also contains thorough information which may serve as a useful refresher for more experienced readers.\n**Pre-requisites:** None\n* * *\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist) roadmap.\n## Inspiration From the Brain\nNeural networks take direct inspiration from the human brain, which is made up of billions of incredibly complex cells called neurons.\n![The Neuron, source](../images/001.png)\nThe Neuron, [source](https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png)\nThe process of thinking within the human brain is the result of communication between neurons. You might receive stimulus in the form of something you saw, then that information is propagated to neurons in the brain via electrochemical signals.\n![eye image generated with Midjourney](../images/002.png)\neye image generated with Midjourney\nThe first neurons in the brain receive that stimulus, then each neuron may choose whether or not to \"fire\" based on how much stimulus it received. \"Firing\", in this case, is a neurons decision to send signals to the neurons it's connected to.\n![Imagine the signal from the eye directly feeds into three neurons, and two decide to fire.](../images/003.png)\nImagine the signal from the eye directly feeds into three neurons, and two decide to fire.\nThen the neurons which those Neurons are connected to may or may not choose to fire.\n![Neurons receive stimulus from previous neurons and then choose whether or not to fire based on the magnitude of the stimulus.](../images/004.png)\nNeurons receive stimulus from previous neurons and then choose whether or not to fire based on the magnitude of the stimulus.\nThus, a \"thought\" can be conceptualized as a large number of neurons choosing to, or not to fire based on the stimulus from other neurons.\nAs one navigates throughout the world, one might have certain thoughts more than another person. A cellist might use some neurons more than a mathematician, for instance.\n![Different tasks require the use of different neurons. Images generated with Midjourney](../images/005.png)\nDifferent tasks require the use of different neurons. Images generated with Midjourney\nWhen we use certain neurons more frequently, their connections become stronger, increasing the intensity of those connections. When we don't use certain neurons, those connections weaken. This general rule has inspired the phrase \"Neurons that fire together, wire together\", and is the high-level quality of the brain which is responsible for the learning process.\n![The process of using certain neurons strengthens their connections.](../images/006.png)\nThe process of using certain neurons strengthens their connections.\nI'm not a neurologist, so of course this is a tremendously simplified description of the brain. However, it's enough to understand the fundamental idea of a neural network.\n## The Intuition of Neural Networks\nNeural networks are, essentially, a mathematically convenient and simplified version of neurons within the brain. A neural network is made up of elements called \"perceptrons\", which are directly inspired by neurons.\n![A perceptron, on the left, vs a neuron, on the right. \\[source\\]\\(https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png\\) 1, source 2](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Vp3uVTyAcixfAlwcZWTTbQ.png)\nA perceptron, on the left, vs a neuron, on the right. [[source](https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png) 1](https://commons.wikimedia.org/wiki/File:ArtificialNeuronModel_english.png), source 2\nPerceptrons take in data, like a neuron does,\n![Perceptrons in AI work with numbers, while Neurons within the brain work with electrochemical signals.](../images/008.png)\nPerceptrons in AI work with numbers, while Neurons within the brain work with electrochemical signals.\naggregate that data, like a neuron does,\n![Perceptrons aggregate numbers to come up with an output, while neurons aggregate electrochemical signals to come up with an output.](../images/009.png)\nPerceptrons aggregate numbers to come up with an output, while neurons aggregate electrochemical signals to come up with an output.\nthen output a signal based on the input, like a neuron does.\n![Perceptrons output numbers, while neurons output electrochemical signals.](../images/010.png)\nPerceptrons output numbers, while neurons output electrochemical signals.\nA neural network can be conceptualized as a big network of these perceptrons, just like the brain is a big network of neurons.\n![A neural network \\(left\\) vs the brain \\(right\\). src1 src2](../images/011.png)\nA neural network (left) vs the brain (right). [src1](https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Artificial_neural_network.svg/560px-Artificial_neural_network.svg.png) [src2](https://www.google.com/url?sa=i&url=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FNeuron&psig=AOvVaw16zt0cwZKdlPi6mQqgRQLT&ust=1736975287779000&source=images&cd=vfe&opi=89978449&ved=0CBQQjRxqFwoTCNi7sKyP9ooDFQAAAAAdAAAAABAE)\nWhen a neuron in the brain fires, it does so as a binary decision. Or, in other words, neurons either fire or they don't. Perceptrons, on the other hand, don't \"fire\" per-se, but output a range of numbers based on the perceptrons input.\n![Perceptrons output a continuous range of numbers, while Neurons either fire or they don't.](../images/012.png)\nPerceptrons output a continuous range of numbers, while Neurons either fire or they don't.\nNeurons within the brain can get away with their relatively simple binary inputs and outputs because thoughts exist over time. Neurons essentially [pulse at different rates](https://www.youtube.com/watch?v=Nxa19uWC_oA), with slower and faster pulses communicating different information.\nSo, neurons have simple inputs and outputs in the form of on or off pulses, but the rate at which they pulse can communicate complex information. Perceptrons only see an input once per pass through the network, but their input and output can be a continuous range of values. If you're familiar with electronics, you might reflect on how this is similar to the relationship between digital and analogue signals.\nThe way the math for a perceptron actually shakes out is pretty simple. A standard neural network consists of a bunch of weights connecting the perceptron's of different layers together.\n![A neural network, with the weights leading into and out of a particular perceptron highlighted.](../images/013.png)\nA neural network, with the weights leading into and out of a particular perceptron highlighted.\nYou can calculate the value of a particular perceptron by adding up all the inputs, multiplied by their respective weights.\n![An example of how the value of a perceptron might be calculated. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\)=-0.09](../images/014.png)\nAn example of how the value of a perceptron might be calculated. (0.3×0.3) + (0.7×0.1) +(-0.5×0.5)=-0.09\nMany Neural Networks also have a \"bias\" associated with each perceptron, which is added to the sum of the inputs to calculate the perceptron's value.\n![An example of how the value of a perceptron might be calculated when a bias term is included in the model. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\) + 0.01 =-0.08](../images/015.png)\nAn example of how the value of a perceptron might be calculated when a bias term is included in the model. (0.3×0.3) + (0.7×0.1) +(-0.5×0.5) + 0.01 =-0.08\nCalculating the output of a neural network, then, is just doing a bunch of addition and multiplication to calculate the value of all the perceptrons.\nSometimes data scientists refer to this general operation as a \"linear projection\", because we're mapping an input into an output via linear operations (addition and multiplication). One problem with this approach is, even if you daisy chain a billion of these layers together, the resulting model will still just be a linear relationship between the input and output because it's all just addition and multiplication.\nThis is a serious problem because not all relationships between an input and output are linear. To get around this, data scientists employ something called an \"activation function\". These are non-linear functions which can be injected throughout the model to, essentially, sprinkle in some non-linearity.\n![Examples of a variety of functions which, given some input, produce some output. The top three are linear, while the bottom three are non-linear.](../images/016.png)\nExamples of a variety of functions which, given some input, produce some output. The top three are linear, while the bottom three are non-linear.\nby interweaving non-linear activation functions between linear projections, neural networks are capable of learning very complex functions,\n![By placing non-linear activation functions within a neural network, neural networks are capable of modeling complex relationships.](../images/017.png)\nBy placing non-linear activation functions within a neural network, neural networks are capable of modeling complex relationships.\nIn AI there are many popular activation functions, but the industry has largely converged on three popular ones: ReLU, Sigmoid, and Softmax, which are used in a variety of different applications. Out of all of them, ReLU is the most common due to its simplicity and ability to generalize to mimic almost any other function.\n![The ReLU activation function, where the output is equal to zero if the input is less than zero, and the output is equal to the input if the input is greater than zero.](../images/018.png)\nThe ReLU activation function, where the output is equal to zero if the input is less than zero, and the output is equal to the input if the input is greater than zero.\nSo, that's the essence of how AI models make predictions. It's a bunch of addition and multiplication with some nonlinear functions sprinkled in between.\nAnother defining characteristic of neural networks is that they can be trained to be better at solving a certain problem, which we'll explore in the next section.\n## Back Propagation\nOne of the fundamental ideas of AI is that you can \"train\" a model. This is done by asking a neural network (which starts its life as a big pile of random data) to do some task. Then, you somehow update the model based on how the model's output compares to a known good answer.\n![The fundamental idea of training a neural network. You give it some data where you know what you want the output to be, compare the neural networks output with your desired result, then use how wrong the neural network was to update the parameters so it's less wrong.](../images/019.png)\nThe fundamental idea of training a neural network. You give it some data where you know what you want the output to be, compare the neural networks output with your desired result, then use how wrong the neural network was to update the parameters so it's less wrong.\nFor this section, let's imagine a neural network with an input layer, a hidden layer, and an output layer.\n![A neural network with two inputs and a single output, with a hidden layer in-between allowing the model to make more complex predictions.](../images/020.png)\nA neural network with two inputs and a single output, with a hidden layer in-between allowing the model to make more complex predictions.\nEach of these layers are connected together with, initially, completely random weights.\n![The neural network, with random weights and biases defined.](../images/021.png)\nThe neural network, with random weights and biases defined.\nAnd we'll use a ReLU activation function on our hidden layer.\n![We'll apply the ReLU activation function to the value of our hidden perceptrons.](../images/022.png)\nWe'll apply the ReLU activation function to the value of our hidden perceptrons.\nLet's say we have some training data, in which the desired output is the average value of the input.\n![An example of the data that we'll be training off of.](../images/023.png)\nAn example of the data that we'll be training off of.\nAnd we pass an example of our training data through the model, generating a prediction.\n![Calculating the value of the hidden layer and output based on the input, including all major intermediary steps.](../images/024.png)\nCalculating the value of the hidden layer and output based on the input, including all major intermediary steps.\nTo make our neural network better at the task of calculating the average of the input, we first compare the predicted output to what our desired output is.\n![The training data has an input of 0.1 and 0.3, and the desired output \\(the average of the input\\) is 0.2. The prediction from the model was -0.1. Thus, the difference between the output and the desired output is 0.3.](../images/025.png)\nThe training data has an input of 0.1 and 0.3, and the desired output (the average of the input) is 0.2. The prediction from the model was -0.1. Thus, the difference between the output and the desired output is 0.3.\nNow that we know that the output should increase in size, we can look back through the model to calculate how our weights and biases might change to promote that change.\nFirst, let's look at the weights leading immediately into the output: w₇, w₈, w₉. Because the output of the third hidden perceptron was -0.46, the activation from ReLU was 0.00.\n![The ultimate, activated output of the third perceptron, 0.00](../images/026.png)\nThe ultimate, activated output of the third perceptron, 0.00\nAs a result, there's no change to w₉ that could result us getting closer to our desired output, because every value of w₉ would result in a change of zero in this particular example.\nThe second hidden neuron, however, does have an activated output which is greater than zero, and thus adjusting w₈ will have an impact on the output for this example.\n![](../images/body-433e00852086943c57da.png)\nThe way we actually calculate how much w₈ should change is by multiplying how much the output should change, times the input to w₈.\n![How we calculate how the weight should change. Here the symbol Δ\\(delta\\) means "change in", so Δw₈ means the "change in w₈"](../images/027.png)\nHow we calculate how the weight should change. Here the symbol Δ(delta) means \"change in\", so Δw₈ means the \"change in w₈\"\nThe easiest explanation of why we do it this way is \"because calculus\", but if we look at how all weights get updated in the last layer, we can form a fun intuition.\n![Calculating how the weights leading into the output should change.](../images/028.png)\nCalculating how the weights leading into the output should change.\nNotice how the two perceptrons that \"fire\" (have an output greater than zero) are updated together. Also, notice how the stronger a perceptrons output is, the more its corresponding weight is updated. This is somewhat similar to the idea that \"Neurons that fire together, wire together\" within the human brain.\nCalculating the change to the output bias is super easy. In fact, we've already done it. Because the bias is how much a perceptrons output should change, the change in the bias is just the changed in the desired output. So, Δb₄=0.3\n![how the bias of the output should be updated.](../images/029.png)\nhow the bias of the output should be updated.\nNow that we've calculated how the weights and bias of the output perceptron should change, we can \"back propagate\" our desired change in output through the model. Let's start with back propagating so we can calculate how we should update w₁.\nFirst, we calculate how the activated output of the of the first hidden neuron should change. We do that by multiplying the change in output by w₇.\n![Calculating how the activated output of the first hidden neuron should have changed by multiplying the desired change in the output by w₇.](../images/body-5f32b00a815b46d229cf.png)\nCalculating how the activated output of the first hidden neuron should have changed by multiplying the desired change in the output by w₇.\nFor values that are greater than zero, ReLU simply multiplies those values by 1. So, for this example, the change we want the un-activated value of the first hidden neuron is equal to the desired change in the activated output\n![How much we want to change the un-activated value of the first hidden perceptron, based on back-propagating from the output.](../images/body-5f5945e256e036f7cc4d.png)\nHow much we want to change the un-activated value of the first hidden perceptron, based on back-propagating from the output.\nRecall that we calculated how to update w₇ based on multiplying it's input by the change in its desired output. We can do the same thing to calculate the change in w₁.\n![Now that we've calculated how the first hidden neuron should change, we can calculate how we should update w₁ the same way we calculated how w₇ should be updated previously.](../images/body-a5aac8c29848f917bab6.png)\nNow that we've calculated how the first hidden neuron should change, we can calculate how we should update w₁ the same way we calculated how w₇ should be updated previously.\nIt's important to note, we're not actually updating any of the weights or biases throughout this process. Rather, we're taking a tally of how we should update each parameter, assuming no other parameters are updated.\nSo, we can do those calculations to calculate all parameter changes.\n![By back propagating through the model, using a combination of values from the forward passes and desired changes from the backward pass at various points of the model, we can calculate how all parameters should change](../images/body-5db53156dd46c9c1dab1.png)\nBy back propagating through the model, using a combination of values from the forward passes and desired changes from the backward pass at various points of the model, we can calculate how all parameters should change\nA fundamental idea of back propagation is called \"Learning Rate\", which concerns the size of the changes we make to neural networks based on a particular batch of data. To explain why this is important, I'd like to use an analogy.\nImagine you went outside one day, and everyone wearing a hat gave you a funny look. You probably don't want to jump to the conclusion that `wearing hat = funny look` , but you might be a bit skeptical of people wearing hats. After three, four, five days, a month, or even a year, if it seems like the vast majority of people wearing hats are giving you a funny look, you may start considering that a strong trend.\nSimilarly, when we train a neural network, we don't want to completely change how the neural network thinks based on a single training example. Rather, we want each batch to only incrementally change how the model thinks. As we expose the model to many examples, we would hope that the model would learn important trends within the data.\nAfter we've calculated how each parameter should change as if it were the only parameter being updated, we can multiply all those changes by a small number, like `0.001` , before applying those changes to the parameters. This small number is commonly referred to as the \"learning rate\", and the exact value it should have is dependent on the model we're training on. This effectively scales down our adjustments before applying them to the model.\nAt this point we covered pretty much everything one would need to know to implement a neural network. Let's give it a shot!\n![Join IAEE](../images/body-45f96194eb609b01245e.jpeg)\n[Join IAEE](https://iaee.substack.com/)\n## Implementing a Neural Network from Scratch\nTypically, a data scientist would just use a library like `PyTorch` to implement a neural network in a few lines of code, but we'll be defining a neural network from the ground up using NumPy, a numerical computing library.\nFirst, let's start with a way to define the structure of the neural network.\npython\n\n```\n\"\"\"Blocking out the structure of the Neural Network\n\"\"\"\n\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\nprint('weight dimensions:')\nfor w in model.weights:\n    print(w.shape)\n\nprint('nbias dimensions:')\nfor b in model.biases:\n    print(b.shape)\n```\n\n![The weight and bias matrix defined in a sample neural network.](../images/body-fe7b89890669846c57ea.png)\nThe weight and bias matrix defined in a sample neural network.\nWhile we typically draw neural networks as a dense web in reality we represent the weights between their connections as matrices. This is convenient because matrix multiplication, then, is equivalent to passing data through a neural network.\n![Thinking of a dense network as weighted connections on the left, and as matrix multiplication on the right. On the right hand side diagram, the vector on the left would be the input, the matrix in the center would be the weight matrix, and the vector on the right would be the output. Only a portion of values are included for readability. From my article on LoRA.](../images/body-39e5089ad8315e01e205.png)\nThinking of a dense network as weighted connections on the left, and as matrix multiplication on the right. On the right hand side diagram, the vector on the left would be the input, the matrix in the center would be the weight matrix, and the vector on the right would be the output. Only a portion of values are included for readability. From my article on [LoRA](https://medium.com/towards-data-science/lora-intuitively-and-exhaustively-explained-e944a6bff46b).\nWe can make our model make a prediction based on some input by passing the input through each layer.\npython\n\n```\n\"\"\"Implementing the Forward Pass\n\"\"\"\n\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        #implementing the relu activation function\n        return np.maximum(0, x)\n\n    def forward(self, X):\n        #iterating through all layers\n        for W, b in zip(self.weights, self.biases):\n\n            #applying the weight and bias of the layer\n            X = np.dot(X, W) + b\n\n            #doing ReLU for all but the last layer\n            if W is not self.weights[-1]:\n                X = self.relu(X)\n\n        #returning the result\n        return X\n\n    def predict(self, X):\n        y = self.forward(X)\n        return y.flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Generate predictions\nprediction = model.predict(np.array([0.1,0.2]))\nprint(prediction)\n```\n\n![the result of passing our data through the model. Our model is randomly defined, so this isn't a useful prediction, but it confirms that the model is working.](../images/body-dc606751352fd4bdd235.png)\nthe result of passing our data through the model. Our model is randomly defined, so this isn't a useful prediction, but it confirms that the model is working.\nWe need to be able to train this model, and to do that we'll first need a problem to train the model on. I defined a random function that takes in two inputs and results in an output:\npython\n\n```\n\"\"\"Defining what we want the model to learn\n\"\"\"\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n# Define a random function with two inputs\ndef random_function(x, y):\n    return (np.sin(x) + x * np.cos(y) + y + 3**(x/3))\n\n# Generate a grid of x and y values\nx = np.linspace(-10, 10, 100)\ny = np.linspace(-10, 10, 100)\nX, Y = np.meshgrid(x, y)\n\n# Compute the output of the random function\nZ = random_function(X, Y)\n\n# Create a 2D plot\nplt.figure(figsize=(8, 6))\ncontour = plt.contourf(X, Y, Z, cmap='viridis')\nplt.colorbar(contour, label='Function Value')\nplt.title('2D Plot of Objective Function')\nplt.xlabel('X-axis')\nplt.ylabel('Y-axis')\nplt.show()\n```\n\n![The modeling objective. Given two inputs \\(here plotted as x and y\\), the model needs to predict an output \\(here represented as color\\). This is a completely arbitrary function](../images/body-894ff02482ae5551d5e5.png)\nThe modeling objective. Given two inputs (here plotted as x and y), the model needs to predict an output (here represented as color). This is a completely arbitrary function\nIn the real world we wouldn't know the underlying function. We can mimic that reality by creating a dataset consisting of random points:\npython\n\n```\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\n# Define a random function with two inputs\ndef random_function(x, y):\n    return (np.sin(x) + x * np.cos(y) + y + 3**(x/3))\n\n# Define the number of random samples to generate\nn_samples = 1000\n\n# Generate random X and Y values within a specified range\nx_min, x_max = -10, 10\ny_min, y_max = -10, 10\n\n# Generate random values for X and Y\nX_random = np.random.uniform(x_min, x_max, n_samples)\nY_random = np.random.uniform(y_min, y_max, n_samples)\n\n# Evaluate the random function at the generated X and Y values\nZ_random = random_function(X_random, Y_random)\n\n# Create a dataset\ndataset = pd.DataFrame({\n    'X': X_random,\n    'Y': Y_random,\n    'Z': Z_random\n})\n\n# Display the dataset\nprint(dataset.head())\n\n# Create a 2D scatter plot of the sampled data\nplt.figure(figsize=(8, 6))\nscatter = plt.scatter(dataset['X'], dataset['Y'], c=dataset['Z'], cmap='viridis', s=10)\nplt.colorbar(scatter, label='Function Value')\nplt.title('Scatter Plot of Randomly Sampled Data')\nplt.xlabel('X-axis')\nplt.ylabel('Y-axis')\nplt.show()\n```\n\n![This is the data we'll be training on to try to learn our function.](../images/body-3f09aab8f8212dcd71f9.png)\nThis is the data we'll be training on to try to learn our function.\nRecall that the back propagation algorithm updates parameters based on what happens in a forward pass. So, before we implement backpropagation itself, let's keep track of a few important values in the forward pass: The inputs and outputs of each perceptron throughout the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        #keeping track of these values in this code block\n        #so we can observe them\n        self.perceptron_inputs = None\n        self.perceptron_outputs = None\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    def forward(self, X):\n        self.perceptron_inputs = [X]\n        self.perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(self.perceptron_inputs[-1], W) + b\n            self.perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            self.perceptron_inputs.append(A)\n\n        return self.perceptron_inputs, self.perceptron_outputs\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Generate predictions\nprediction = model.predict(np.array([0.1,0.2]))\n\n#looking through critical optimization values\nfor i, (inpt, outpt) in enumerate(zip(model.perceptron_inputs, model.perceptron_outputs[:-1])):\n    print(f'layer {i}')\n    print(f'input: {inpt.shape}')\n    print(f'output: {outpt.shape}')\n    print('')\n\nprint('Final Output:')\nprint(model.perceptron_outputs[-1].shape)\n```\n\n![The values throughout various layers of the model as a result of the forward pass. This will allow us to compute the necessary changes to update the model.](../images/body-672a8a9c3cce44fe0ce1.png)\nThe values throughout various layers of the model as a result of the forward pass. This will allow us to compute the necessary changes to update the model.\nNow that we have a record stored of critical intermediary value within the network, we can use those values, along with the error of a model for a particular prediction, to calculate the changes we should make to the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    @staticmethod\n    def relu_as_weights(x):\n        return (x > 0).astype(float)\n\n    def forward(self, X):\n        perceptron_inputs = [X]\n        perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(perceptron_inputs[-1], W) + b\n            perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            perceptron_inputs.append(A)\n\n        return perceptron_inputs, perceptron_outputs\n\n    def backward(self, perceptron_inputs, perceptron_outputs, target):\n        weight_changes = []\n        bias_changes = []\n\n        m = len(target)\n        dA = perceptron_inputs[-1] - target.reshape(-1, 1)  # Output layer gradient\n\n        for i in reversed(range(len(self.weights))):\n            dZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n            dW = np.dot(perceptron_inputs[i].T, dZ) / m\n            db = np.sum(dZ, axis=0, keepdims=True) / m\n            weight_changes.append(dW)\n            bias_changes.append(db)\n\n            if i > 0:\n                dA = np.dot(dZ, self.weights[i].T)\n\n        return list(reversed(weight_changes)), list(reversed(bias_changes))\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n#defining a sample input and target output\ninput = np.array([[0.1,0.2]])\ndesired_output = np.array([0.5])\n\n#doing forward and backward pass to calculate changes\nperceptron_inputs, perceptron_outputs = model.forward(input)\nweight_changes, bias_changes = model.backward(perceptron_inputs, perceptron_outputs, desired_output)\n\n#smaller numbers for printing\nnp.set_printoptions(precision=2)\n\nfor i, (layer_weights, layer_biases, layer_weight_changes, layer_bias_changes)\nin enumerate(zip(model.weights, model.biases, weight_changes, bias_changes)):\n    print(f'layer {i}')\n    print(f'weight matrix: {layer_weights.shape}')\n    print(f'weight matrix changes: {layer_weight_changes.shape}')\n    print(f'bias matrix: {layer_biases.shape}')\n    print(f'bias matrix changes: {layer_bias_changes.shape}')\n    print('')\n\nprint('The weight and weight change matrix of the second layer:')\nprint('weight matrix:')\nprint(model.weights[1])\nprint('change matrix:')\nprint(weight_changes[1])\n```\n\n![](../images/body-e504bb15c3bb047018b3.png)\nThis is probably the most complex implementation step, so I want to take a moment to dig through some of the details. The fundamental idea is exactly as we described in previous sections. We're iterating over all layers, from back to front, and calculating what change to each weight and bias would result in a better output.\npython\n\n```\n# calculating output error\ndA = perceptron_inputs[-1] - target.reshape(-1, 1)\n\n#a scaling factor for the batch size.\n#you want changes to be an average across all batches\n#so we divide by m once we've aggregated all changes.\nm = len(target)\n\nfor i in reversed(range(len(self.weights))):\n  dZ = dA #simplified for now\n\n  # calculating change to weights\n  dW = np.dot(perceptron_inputs[i].T, dZ) / m\n  # calculating change to bias\n  db = np.sum(dZ, axis=0, keepdims=True) / m\n\n  # keeping track of required changes\n  weight_changes.append(dW)\n  bias_changes.append(db)\n  ...\n```\n\nCalculating the change to bias is pretty straight forward. If you look at how the output of a given neuron should have impacted all future neurons, you can add up all those values (which are both positive and negative) to get an idea of if the neuron should be biased in a positive or negative direction.\nThe way we calculate the change to weights, by using matrix multiplication, is a bit more mathematically complex.\nini\n\n```\ndW = np.dot(perceptron_inputs[i].T, dZ) / m\n```\n\nBasically, this line says that the change in the weight should be equal to the value going into the perceptron, times how much the output should have changed. If a perceptron had a big input, the change to its outgoing weights should be a large magnitude, if the perceptron had a small input, the change to its outgoing weights will be small. Also, if a weight points towards an output which should change a lot, the weight should change a lot.\nThere is another line we should discuss in our back propagation implement.\nlua\n\n```\ndZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n```\n\nIn this particular network, there are activation functions throughout the network, following all but the final output. When we do back propagation, we need to back-propagate through these activation functions so that we can update the neurons which lie before them. We do this for all but the last layer, which doesn't have an activation function, which is why `dZ = dA if i == len(self.weights) - 1` .\nIn fancy math speak we would call this a derivative, but because I don't want to get into calculus, I called the function `relu_as_weights` . Basically, we can treat each of our ReLU activations as something like a tiny neural network, who's weight is a function of the input. If the input of the ReLU activation function is less than zero, then that's like passing that input through a neural network with a weight of zero. If the input of ReLU is greater than zero, then that's like passing the input through a neural netowork with a weight of one.\n![Recall the ReLU activation function.](../images/018.png)\nRecall the ReLU activation function.\nThis is exactly what the `relu_as_weights` function does.\npython\n\n```\ndef relu_as_weights(x):\n        return (x > 0).astype(float)\n```\n\nUsing this logic we can treat back propagating through ReLU just like we back propagate through the rest of the neural network.\nAgain, I'll be covering this concept from a more robust mathematical prospective soon, but that's the essential idea from a conceptual perspective.\nNow that we have the forward and backward pass implemented, we can implement training the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    @staticmethod\n    def relu_as_weights(x):\n        return (x > 0).astype(float)\n\n    def forward(self, X):\n        perceptron_inputs = [X]\n        perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(perceptron_inputs[-1], W) + b\n            perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            perceptron_inputs.append(A)\n\n        return perceptron_inputs, perceptron_outputs\n\n    def backward(self, perceptron_inputs, perceptron_outputs, y_true):\n        weight_changes = []\n        bias_changes = []\n\n        m = len(y_true)\n        dA = perceptron_inputs[-1] - y_true.reshape(-1, 1)  # Output layer gradient\n\n        for i in reversed(range(len(self.weights))):\n            dZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n            dW = np.dot(perceptron_inputs[i].T, dZ) / m\n            db = np.sum(dZ, axis=0, keepdims=True) / m\n            weight_changes.append(dW)\n            bias_changes.append(db)\n\n            if i > 0:\n                dA = np.dot(dZ, self.weights[i].T)\n\n        return list(reversed(weight_changes)), list(reversed(bias_changes))\n\n    def update_weights(self, weight_changes, bias_changes, lr):\n        for i in range(len(self.weights)):\n            self.weights[i] -= lr * weight_changes[i]\n            self.biases[i] -= lr * bias_changes[i]\n\n    def train(self, X, y, epochs, lr=0.01):\n        for epoch in range(epochs):\n            perceptron_inputs, perceptron_outputs = self.forward(X)\n            weight_changes, bias_changes = self.backward(perceptron_inputs, perceptron_outputs, y)\n            self.update_weights(weight_changes, bias_changes, lr)\n\n            if epoch % 20 == 0 or epoch == epochs - 1:\n                loss = np.mean((perceptron_inputs[-1].flatten() - y) ** 2)  # MSE\n                print(f\"EPOCH {epoch}: Loss = {loss:.4f}\")\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n```\n\nThe `train` function:\n  * iterates through all the data some number of times (defined by `epoch` )\n  * passes the data through a forward pass\n  * calculates how the weights and biases should change\n  * updates the weights and biases, by scaling their changes by the learning rate ( `lr` )\n\n\nAnd thus we've implemented a neural network! Let's train it.\n## Training and Evaluating the Neural Network.\nRecall that we defined an arbitrary 2D function we wanted to learn how to emulate,\n![](../images/body-894ff02482ae5551d5e5.png)\nand we sampled that space with some number of points, which we're using to train the model.\n![](../images/body-04b5ffea74ad756d758c.png)\nBefore feeding this data into our model, it's vital that we first \"normalize\" the data. Certain values of the dataset are very small or very large, which can make training a neural network very difficult. Values within the neural network can quickly grow to absurdly large values, or diminish to zero, which can inhibit training. Normalization squashes all of our inputs, and our desired outputs, into a more reasonable range averaging around zero with a standardized distribution called a \"normal\" distribution.\nini\n\n```\n# Flatten the data\nX_flat = X.flatten()\nY_flat = Y.flatten()\nZ_flat = Z.flatten()\n\n# Stack X and Y as input features\ninputs = np.column_stack((X_flat, Y_flat))\noutputs = Z_flat\n\n# Normalize the inputs and outputs\ninputs_mean = np.mean(inputs, axis=0)\ninputs_std = np.std(inputs, axis=0)\noutputs_mean = np.mean(outputs)\noutputs_std = np.std(outputs)\n\ninputs = (inputs - inputs_mean) / inputs_std\noutputs = (outputs - outputs_mean) / outputs_std\n```\n\nIf we want to get back predictions in the actual range of data from our original dataset, we can use these values to essentially \"un-squash\" the data.\nOnce we've done that, we can define and train our model.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=2000, lr=0.001)\n```\n\n![As can be seen, the value of loss is going down consistently, implying the model is improving.](../images/body-8be8441ac37a9e97ec54.png)\nAs can be seen, the value of loss is going down consistently, implying the model is improving.\nThen we can visualize the output of the neural network's prediction vs the actual function.\npython\n\n```\nimport matplotlib.pyplot as plt\n\n# Reshape predictions to grid format for visualization\nZ_pred = model.predict(inputs) * outputs_std + outputs_mean\nZ_pred = Z_pred.reshape(X.shape)\n\n# Plot comparison of the true function and the model predictions\nfig, axes = plt.subplots(1, 2, figsize=(14, 6))\n\n# Plot the true function\naxes[0].contourf(X, Y, Z, cmap='viridis')\naxes[0].set_title(\"True Function\")\naxes[0].set_xlabel(\"X-axis\")\naxes[0].set_ylabel(\"Y-axis\")\naxes[0].colorbar = plt.colorbar(axes[0].contourf(X, Y, Z, cmap='viridis'), ax=axes[0], label=\"Function Value\")\n\n# Plot the predicted function\naxes[1].contourf(X, Y, Z_pred, cmap='plasma')\naxes[1].set_title(\"NN Predicted Function\")\naxes[1].set_xlabel(\"X-axis\")\naxes[1].set_ylabel(\"Y-axis\")\naxes[1].colorbar = plt.colorbar(axes[1].contourf(X, Y, Z_pred, cmap='plasma'), ax=axes[1], label=\"Function Value\")\n\nplt.tight_layout()\nplt.show()\n```\n\n![](../images/body-f8bed791ef176da3990d.png)\nThis did an ok job, but not as great as we might like. This is where a lot of data scientists spend their time, and there are a ton of approaches to making a neural network fit a certain problem better. Some obvious ones are:\n  * use more data\n  * play around with the learning rate\n  * train for more epochs\n  * change the structure of the model\n\n\nIt's pretty easy for us to crank up the amount of data we're training on. Let's see where that leads us. Here I'm sampling our dataset 10,000 times, which is 10x more training samples than our previous dataset.\n![](../images/body-53bd3af41cede2c618ac.png)\nAnd then I trained the model just like before, except this time it took a lot longer because each epoch now analyses 10,000 samples rather than 1,000.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=2000, lr=0.001)\n```\n\n![](../images/body-f521926776cf9022867e.png)\nI then rendered the output of this model, the same way I did before, but it didn't really look like the output got much better.\n![](../images/body-90739b15c2d72b8cc4a8.png)\nLooking back at the loss output from training, it seems like the loss is still steadily declining. Maybe I just need to train for longer. Let's try that.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=4000, lr=0.001)\n```\n\n![](../images/body-3664d506356f892eed52.png)\nThe results seem to be a bit better, but they aren't' amazing.\n![](../images/body-917f1e489f8d8c464b00.png)\nI'll spare you the details. I ran this a few times, and I got some decent results, but never anything 1 to 1. I'll be covering some more advanced approaches data scientists use, like annealing and dropout, in future articles which will result in a more consistent and better output. Still, though, we made a neural network from scratch and trained it to do something, and it did a decent job! Pretty neat!\n## Conclusion\nIn this article we avoided calculus like the plague while simultaneously forging an understanding of Neural Networks. We explored their theory, a little bit about the math, the idea of back propagation, and then implemented a neural network from scratch. We then applied a neural network to a toy problem, and explored some of the simple ideas data scientists employ to actually train neural networks to be good at things.\nIn future articles we'll explore a few more advanced approaches to Neural Networks, so stay tuned! For now, you might be interested in a more thorough analysis of Gradients, the fundamental math behind back propagation.\n> [**What Are Gradients, and Why Do They Explode?**](https://towardsdatascience.com/what-are-gradients-and-why-do-they-explode-add23264d24b)\nYou might also be interested in this article, which covers training a neural network using more conventional data science tools like PyTorch.\n> [**AI for the Absolute Novice - Intuitively and Exhaustively Explained**](https://towardsdatascience.com/ai-for-the-absolute-novice-intuitively-and-exhaustively-explained-7b353a31e6d7)\n## Join Intuitively and Exhaustively Explained\nAt IAEE you can find:\n  * Long form content, like the article you just read\n  * Conceptual breakdowns of some of the most cutting-edge AI topics\n  * By-Hand walkthroughs of critical mathematical operations in AI\n  * Practical tutorials and explainers\n\n\n![Join IAEE](../images/body-19879eaea2fcf96a966c.jpeg)\n[Join IAEE](https://iaee.substack.com/)\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-a5fed8e6ca1d5b1466fe.jpeg)\nWritten By\n## [Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\n[See all from Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[AI](https://towardsdatascience.com/tag/ai/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Neural Networks](https://towardsdatascience.com/tag/neural-networks/)[Software Development](https://towardsdatascience.com/tag/software-development/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&title=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&title=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&text=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](../images/body-977ccc5d3867bd55d9e7.jpg)](https://towardsdatascience.com/choosing-the-right-tool-for-your-next-data-science-web-application-3d7ec8293d61/)\n## [Choosing the Right Tool for Your Next Data Science Web Application](https://towardsdatascience.com/choosing-the-right-tool-for-your-next-data-science-web-application-3d7ec8293d61/)\n[Programming](https://towardsdatascience.com/category/programming/)\nFlask, Django, Streamlit: Three options for data scientists getting into web development.\n[Murtaza Ali](https://towardsdatascience.com/author/murtaza5152-ali/)\nJanuary 31, 20237 min read\n  * [![](../images/body-5228ebb6633a7aa07706.png)](https://towardsdatascience.com/towards-removing-gender-bias-in-writing-f1307d0a2b71/)\n## [Towards Removing Gender Bias in Writing](https://towardsdatascience.com/towards-removing-gender-bias-in-writing-f1307d0a2b71/)\n[NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/)\nLearn how to understand the amount of gender bias in your daily consumption of literature and avoid it in your own writing.\n[Srihitha Pallapothula](https://towardsdatascience.com/author/spallapothula2004/)\nSeptember 30, 202111 min read\n  * [![](../images/body-19f1c63429a1349c37cd.png)](https://towardsdatascience.com/normalization-vs-standardization-which-one-is-better-f29e043a57eb/)\n## [Normalization vs Standardization, which one is better](https://towardsdatascience.com/normalization-vs-standardization-which-one-is-better-f29e043a57eb/)\n[Programming](https://towardsdatascience.com/category/programming/)\nIn this tutorial let us see which one is the best feature engineering technique of them all.\n[Tanu N Prabhu](https://towardsdatascience.com/author/tanunprabhu95/)\nApril 22, 20205 min read\n  * [![](../images/body-0c5ee854975a540d975c.jpg)](https://towardsdatascience.com/how-to-improve-ar-predictions-using-arma-for-time-series-analysis-483b84984490/)\n## [How To Improve AR Predictions Using ARMA For Time Series Analysis](https://towardsdatascience.com/how-to-improve-ar-predictions-using-arma-for-time-series-analysis-483b84984490/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nLearn to use an ARMA model to make better predictions than an AR model.\n[Jacob Toftgaard Rasmussen](https://towardsdatascience.com/author/jacobtoftgaardrasmussen/)\nJanuary 14, 20225 min read\n  * [![](../images/body-7719e82bbb97733c4b99.png)](https://towardsdatascience.com/causal-machine-learning-what-can-we-accomplish-with-a-single-theorem-ea81432f1fac/)\n## [Causal Machine Learning: What Can We Accomplish with a Single Theorem?](https://towardsdatascience.com/causal-machine-learning-what-can-we-accomplish-with-a-single-theorem-ea81432f1fac/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nExploring and exploiting the seemingly innocent theorem behind Double Machine Learning\n[Harrison Hoffman](https://towardsdatascience.com/author/harrisonfhoffman/)\nMarch 30, 202429 min read\n  * [![](../images/body-c81c2de274002e65949c.jpeg)](https://towardsdatascience.com/the-unknown-features-of-pythons-operator-module-1ad9075d9536/)\n## [The Unknown Features of Python's Operator Module](https://towardsdatascience.com/the-unknown-features-of-pythons-operator-module-1ad9075d9536/)\n[Programming](https://towardsdatascience.com/category/programming/)\nMake your code faster, concise, more readable and more functional with help of Python's little known operator module\n[Martin Heinz](https://towardsdatascience.com/author/martin.heinz/)\nJuly 26, 20216 min read\n  * [![](../images/body-043d49516d3dbde285e1.jpg)](https://towardsdatascience.com/stop-being-data-driven-583b5e7abe7b/)\n## [Stop Being Data-Driven](https://towardsdatascience.com/stop-being-data-driven-583b5e7abe7b/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nWhy we are fooled by data and how to stop it\n[Cai Parry-Jones](https://towardsdatascience.com/author/caiparryjones96/)\nAugust 29, 202411 min read\n  * [![](../images/body-cbefa9eb6e6731e0c8c1.jpeg)](https://towardsdatascience.com/ai-prototypes-for-clients-as-a-marketing-strategy-70de404d27b5/)\n## [AI Prototypes For Clients As A Marketing Strategy](https://towardsdatascience.com/ai-prototypes-for-clients-as-a-marketing-strategy-70de404d27b5/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWhere is the line? Isn't the goal to get paid?\n[Mathieu Lemay](https://towardsdatascience.com/author/lsci/)\nFebruary 14, 20185 min read\n  * [![](../images/body-3883bd1ddfa29151abd0.jpg)](https://towardsdatascience.com/how-to-test-pandas-etl-data-pipeline-e49fb5dac4ce/)\n## [How to Test Pandas ETL Data Pipeline](https://towardsdatascience.com/how-to-test-pandas-etl-data-pipeline-e49fb5dac4ce/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTest your Pandas ETL data pipeline with Great Expectations\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nSeptember 19, 20228 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\nword word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word\n  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1\n####  Recommended Articles \nClose\n",
  "metadata": {
    "title": "Neural Networks - Intuitively and Exhaustively Explained | Towards Data Science",
    "description": "An in-depth exploration of the most fundamental architecture in modern AI.",
    "keywords": null,
    "author": "Daniel Warfield",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "Neural Networks - Intuitively and Exhaustively Explained",
    "og:description": "An in-depth exploration of the most fundamental architecture in modern AI.",
    "og:url": "https://towardsdatascience.com/neural-networks-intuitively-and-exhaustively-explained-0153f85c1007/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1B7C0TLxpFO6fKhqXGRe2bg.png",
    "og:image:width": "1024",
    "og:image:height": "1024",
    "og:image:type": "image/png",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "Daniel Warfield",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "25 min read",
    "article:published_time": "2025-02-03T15:02:05.000Z",
    "article:modified_time": "2025-02-03T15:02:05.000Z",
    "article:author": "https://towardsdatascience.com/author/danielwarfield1/"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": "",
    "retried": true
  },
  "code_blocks": [
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_000.txt",
      "chars": 812
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_001.txt",
      "chars": 1341
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_002.txt",
      "chars": 654
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_003.txt",
      "chars": 1088
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_004.txt",
      "chars": 1911
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_005.txt",
      "chars": 3107
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_006.txt",
      "chars": 573
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_007.txt",
      "chars": 44
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_008.txt",
      "chars": 92
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_009.txt",
      "chars": 61
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_010.txt",
      "chars": 2727
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_011.txt",
      "chars": 444
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_012.txt",
      "chars": 246
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_013.txt",
      "chars": 891
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_014.txt",
      "chars": 246
    },
    {
      "language": "txt",
      "file": "attachments/5361b27a4c83/code/block_015.txt",
      "chars": 246
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1B7C0TLxpFO6fKhqXGRe2bg.png",
      "bytes": 1334902,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/000.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1B7C0TLxpFO6fKhqXGRe2bg.png",
      "alt": ""The Thinking Part" by Daniel Warfield using MidJourney. All images by the author unless otherwise specified. Article originally made available on Intuitively and Exhaustively Explained.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0H6bVaEAMKw2WlsXB.png",
      "bytes": 939443,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/001.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0H6bVaEAMKw2WlsXB.png",
      "alt": "The Neuron, source",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xfX_DPDEwe3sUE7N1wIZvw.png",
      "bytes": 99720,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/002.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xfX_DPDEwe3sUE7N1wIZvw.png",
      "alt": "eye image generated with Midjourney",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11_-vnr8Ax_TqknfAYA2A-w.png",
      "bytes": 108223,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/003.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11_-vnr8Ax_TqknfAYA2A-w.png",
      "alt": "Imagine the signal from the eye directly feeds into three neurons, and two decide to fire.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1fIUUxfy2BMODCRmi1D3SBw.png",
      "bytes": 120403,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/004.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1fIUUxfy2BMODCRmi1D3SBw.png",
      "alt": "Neurons receive stimulus from previous neurons and then choose whether or not to fire based on the magnitude of the stimulus.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v_6qjOhyH3DT0rEj95_zRg.png",
      "bytes": 178641,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/005.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v_6qjOhyH3DT0rEj95_zRg.png",
      "alt": "Different tasks require the use of different neurons. Images generated with Midjourney",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13opdx0rZVIavS3hd-FtJbw.png",
      "bytes": 201934,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/006.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13opdx0rZVIavS3hd-FtJbw.png",
      "alt": "The process of using certain neurons strengthens their connections.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Ixk7vi9afHRoRx5dG_E37A.png",
      "bytes": 23465,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/008.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Ixk7vi9afHRoRx5dG_E37A.png",
      "alt": "Perceptrons in AI work with numbers, while Neurons within the brain work with electrochemical signals.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1M4Hg_rE7WmLFqpMjfg5P3Q.png",
      "bytes": 31065,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/009.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1M4Hg_rE7WmLFqpMjfg5P3Q.png",
      "alt": "Perceptrons aggregate numbers to come up with an output, while neurons aggregate electrochemical signals to come up with an output.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1DHnCzMRpcBASTtxFMeAuFg.png",
      "bytes": 33362,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/010.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1DHnCzMRpcBASTtxFMeAuFg.png",
      "alt": "Perceptrons output numbers, while neurons output electrochemical signals.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1-7K5ccbzd9_UO6UIRzxbOA.png",
      "bytes": 665357,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/011.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1-7K5ccbzd9_UO6UIRzxbOA.png",
      "alt": "A neural network \\(left\\) vs the brain \\(right\\). src1 src2",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11snQ4GLqxc-6tHjXaF6TtA.png",
      "bytes": 28408,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/012.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11snQ4GLqxc-6tHjXaF6TtA.png",
      "alt": "Perceptrons output a continuous range of numbers, while Neurons either fire or they don't.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1cLcbZkQ4KCmyqsoIsjEQCg.png",
      "bytes": 61248,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/013.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1cLcbZkQ4KCmyqsoIsjEQCg.png",
      "alt": "A neural network, with the weights leading into and out of a particular perceptron highlighted.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Sd0t4WYtLZmhvF6w6VcQ6A.png",
      "bytes": 68607,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/014.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Sd0t4WYtLZmhvF6w6VcQ6A.png",
      "alt": "An example of how the value of a perceptron might be calculated. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\)=-0.09",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1gTOvMB96VO5qDn-VI7Me_w.png",
      "bytes": 142932,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/015.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1gTOvMB96VO5qDn-VI7Me_w.png",
      "alt": "An example of how the value of a perceptron might be calculated when a bias term is included in the model. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\) + 0.01 =-0.08",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1q-oox60ZsuYVYbZkp2SpEg.png",
      "bytes": 43125,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/016.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1q-oox60ZsuYVYbZkp2SpEg.png",
      "alt": "Examples of a variety of functions which, given some input, produce some output. The top three are linear, while the bottom three are non-linear.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1m_PdmmIOUeKQZDM59eehNw.png",
      "bytes": 50307,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/017.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1m_PdmmIOUeKQZDM59eehNw.png",
      "alt": "By placing non-linear activation functions within a neural network, neural networks are capable of modeling complex relationships.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1U2_yx_M8XCD93RPOxvg9hQ.png",
      "bytes": 7126,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/018.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1U2_yx_M8XCD93RPOxvg9hQ.png",
      "alt": "The ReLU activation function, where the output is equal to zero if the input is less than zero, and the output is equal to the input if the input is greater than zero.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1JYyCqZhkWAvTaZ7fPhnk9A.png",
      "bytes": 20144,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/019.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1JYyCqZhkWAvTaZ7fPhnk9A.png",
      "alt": "The fundamental idea of training a neural network. You give it some data where you know what you want the output to be, compare the neural networks output with your desired result, then use how wrong the neural network was to update the parameters so it's less wrong.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v2Fwu8wCaSoKiTBBpZknuQ.png",
      "bytes": 16704,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/020.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v2Fwu8wCaSoKiTBBpZknuQ.png",
      "alt": "A neural network with two inputs and a single output, with a hidden layer in-between allowing the model to make more complex predictions.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1UD0Z1G4QBuVvud3Sf8gOWg.png",
      "bytes": 81431,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/021.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1UD0Z1G4QBuVvud3Sf8gOWg.png",
      "alt": "The neural network, with random weights and biases defined.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1pLo_h8SZrJbAjd42Qeyv_A.png",
      "bytes": 95972,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/022.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1pLo_h8SZrJbAjd42Qeyv_A.png",
      "alt": "We'll apply the ReLU activation function to the value of our hidden perceptrons.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NI1jil4RRhvW8m9yPrWTnQ.png",
      "bytes": 9239,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/023.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NI1jil4RRhvW8m9yPrWTnQ.png",
      "alt": "An example of the data that we'll be training off of.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15fKc61r8UC33OQqn5v0Gnw.png",
      "bytes": 86233,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/024.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15fKc61r8UC33OQqn5v0Gnw.png",
      "alt": "Calculating the value of the hidden layer and output based on the input, including all major intermediary steps.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NNXP11WMJkbRzxtcEI7eZQ.png",
      "bytes": 103429,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/025.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NNXP11WMJkbRzxtcEI7eZQ.png",
      "alt": "The training data has an input of 0.1 and 0.3, and the desired output \\(the average of the input\\) is 0.2. The prediction from the model was -0.1. Thus, the difference between the output and the desired output is 0.3.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1dIbrD12PNayTGNhOD75nmg.png",
      "bytes": 70585,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/026.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1dIbrD12PNayTGNhOD75nmg.png",
      "alt": "The ultimate, activated output of the third perceptron, 0.00",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13Whf1AhkWz6clG06cztpCw.png",
      "bytes": 71248,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-433e00852086943c57da.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13Whf1AhkWz6clG06cztpCw.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1aT1An9_PHK2UdYPg4P396g.png",
      "bytes": 93629,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/027.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1aT1An9_PHK2UdYPg4P396g.png",
      "alt": "How we calculate how the weight should change. Here the symbol Δ\\(delta\\) means "change in", so Δw₈ means the "change in w₈"",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1kmHCi-J7Fg3IV6aEynIagQ.png",
      "bytes": 97506,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/028.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1kmHCi-J7Fg3IV6aEynIagQ.png",
      "alt": "Calculating how the weights leading into the output should change.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1yU1vGFGDcYF_bsYpi1EViQ.png",
      "bytes": 63695,
      "reused": true,
      "local_path": "attachments/5361b27a4c83/images/029.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1yU1vGFGDcYF_bsYpi1EViQ.png",
      "alt": "how the bias of the output should be updated.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15bZbQeeglMZ8eNSAMBADJg.png",
      "bytes": 74473,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-5f32b00a815b46d229cf.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15bZbQeeglMZ8eNSAMBADJg.png",
      "alt": "Calculating how the activated output of the first hidden neuron should have changed by multiplying the desired change in the output by w₇.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1MG5zTdstAdlxDYriLA1dMw.png",
      "bytes": 76514,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-5f5945e256e036f7cc4d.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1MG5zTdstAdlxDYriLA1dMw.png",
      "alt": "How much we want to change the un-activated value of the first hidden perceptron, based on back-propagating from the output.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1rb2kMoJK5HkgeBIvWeXYZA.png",
      "bytes": 108773,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-a5aac8c29848f917bab6.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1rb2kMoJK5HkgeBIvWeXYZA.png",
      "alt": "Now that we've calculated how the first hidden neuron should change, we can calculate how we should update w₁ the same way we calculated how w₇ should be updated previously.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15zQm78JT5IjAvQSI0YdEdw.png",
      "bytes": 98243,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-5db53156dd46c9c1dab1.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15zQm78JT5IjAvQSI0YdEdw.png",
      "alt": "By back propagating through the model, using a combination of values from the forward passes and desired changes from the backward pass at various points of the model, we can calculate how all parameters should change",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0SOOewKyrD86bJlSQ.jpeg",
      "bytes": 180281,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-45f96194eb609b01245e.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0SOOewKyrD86bJlSQ.jpeg",
      "alt": "Join IAEE",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Bn8-3lou0sBs8y2cf1sY8g.png",
      "bytes": 10709,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-fe7b89890669846c57ea.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Bn8-3lou0sBs8y2cf1sY8g.png",
      "alt": "The weight and bias matrix defined in a sample neural network.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/02kqALjWcLyiGL9YQ.png",
      "bytes": 146582,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-39e5089ad8315e01e205.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/02kqALjWcLyiGL9YQ.png",
      "alt": "Thinking of a dense network as weighted connections on the left, and as matrix multiplication on the right. On the right hand side diagram, the vector on the left would be the input, the matrix in the center would be the weight matrix, and the vector on the right would be the output. Only a portion of values are included for readability. From my article on LoRA.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/12gnRkuco9SpHeEpKaynhrQ.png",
      "bytes": 1880,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-dc606751352fd4bdd235.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/12gnRkuco9SpHeEpKaynhrQ.png",
      "alt": "the result of passing our data through the model. Our model is randomly defined, so this isn't a useful prediction, but it confirms that the model is working.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11f28YdNHVOI7goP3REmrfA.png",
      "bytes": 82759,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-894ff02482ae5551d5e5.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11f28YdNHVOI7goP3REmrfA.png",
      "alt": "The modeling objective. Given two inputs \\(here plotted as x and y\\), the model needs to predict an output \\(here represented as color\\). This is a completely arbitrary function",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nw8UuI7bSKM_Qth-LYCeMA.png",
      "bytes": 426390,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-3f09aab8f8212dcd71f9.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nw8UuI7bSKM_Qth-LYCeMA.png",
      "alt": "This is the data we'll be training on to try to learn our function.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/17hmH0UrcyzSjmfLAABusZA.png",
      "bytes": 11625,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-672a8a9c3cce44fe0ce1.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/17hmH0UrcyzSjmfLAABusZA.png",
      "alt": "The values throughout various layers of the model as a result of the forward pass. This will allow us to compute the necessary changes to update the model.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1J7TFKgd0gBikRMuiMin2Rw.png",
      "bytes": 61537,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-e504bb15c3bb047018b3.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1J7TFKgd0gBikRMuiMin2Rw.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NESTsjN_9dOKVmWgP12ZGA.png",
      "bytes": 391684,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-04b5ffea74ad756d758c.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NESTsjN_9dOKVmWgP12ZGA.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nXquA-yHCZrfdwSHSWwvzw.png",
      "bytes": 60716,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-8be8441ac37a9e97ec54.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nXquA-yHCZrfdwSHSWwvzw.png",
      "alt": "As can be seen, the value of loss is going down consistently, implying the model is improving.",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1WD5ABUHvbEoC2Bg8Z-Ob6w.png",
      "bytes": 196265,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-f8bed791ef176da3990d.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1WD5ABUHvbEoC2Bg8Z-Ob6w.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1V9NO7sBco4qsMgpzPU6e9w.png",
      "bytes": 903061,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-53bd3af41cede2c618ac.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1V9NO7sBco4qsMgpzPU6e9w.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/14RK1Zw7_QvVMncehuB5aLg.png",
      "bytes": 46859,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-f521926776cf9022867e.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/14RK1Zw7_QvVMncehuB5aLg.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xi2uWp2nG_RW745IEOplPw.png",
      "bytes": 196400,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-90739b15c2d72b8cc4a8.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xi2uWp2nG_RW745IEOplPw.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1LuD6WcQFUxI3-kLmOxZUkw.png",
      "bytes": 48792,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-3664d506356f892eed52.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1LuD6WcQFUxI3-kLmOxZUkw.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1QOcHdFMieEoEKvOdPjzMzg.png",
      "bytes": 193734,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-917f1e489f8d8c464b00.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1QOcHdFMieEoEKvOdPjzMzg.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0tBoYEtqCyiht-7gI.jpeg",
      "bytes": 83081,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-19879eaea2fcf96a966c.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0tBoYEtqCyiht-7gI.jpeg",
      "alt": "Join IAEE",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/1787657187493_78jlbn.jpeg",
      "bytes": 76838,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-a5fed8e6ca1d5b1466fe.jpeg",
      "src": "https://assets.insightmediagroup.io/media/1787657187493_78jlbn.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/01/0ZOz1JOTOg2dKz00B-scaled.jpg",
      "bytes": 332469,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-977ccc5d3867bd55d9e7.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2023/01/0ZOz1JOTOg2dKz00B-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/0wuc6dhYpBqr8YvNn.png",
      "bytes": 2043999,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-5228ebb6633a7aa07706.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/0wuc6dhYpBqr8YvNn.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/04/1sS20G4rBJLNNwCzBs0g7gA.png",
      "bytes": 20095,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-19f1c63429a1349c37cd.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2020/04/1sS20G4rBJLNNwCzBs0g7gA.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/01/05XVjt3c2YulvMs43-scaled.jpg",
      "bytes": 458379,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-0c5ee854975a540d975c.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/01/05XVjt3c2YulvMs43-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/03/1CnM4oJOQBSt33oLgLSLMog.png",
      "bytes": 1604452,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/5361b27a4c83/images/body-7719e82bbb97733c4b99.png",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/03/1CnM4oJOQBSt33oLgLSLMog.png",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/1ko3qDrEmcYHOwvdvj0wBnA-scaled.jpeg",
      "bytes": 324138,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-c81c2de274002e65949c.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/1ko3qDrEmcYHOwvdvj0wBnA-scaled.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/0-DMUzqsJxapVIqS9-scaled.jpg",
      "bytes": 324545,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-043d49516d3dbde285e1.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/0-DMUzqsJxapVIqS9-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2018/02/1i18hfdYba3iEiY6MCSsiAw.jpeg",
      "bytes": 138863,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-cbefa9eb6e6731e0c8c1.jpeg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2018/02/1i18hfdYba3iEiY6MCSsiAw.jpeg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    },
    {
      "status": "ok",
      "url": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/09/0jMeWfwsZhRDhecGB-scaled.jpg",
      "bytes": 556490,
      "reused": false,
      "content_type": "image/jpeg",
      "local_path": "attachments/5361b27a4c83/images/body-3883bd1ddfa29151abd0.jpg",
      "src": "https://assets.insightmediagroup.io/media/wp-content/uploads/2022/09/0jMeWfwsZhRDhecGB-scaled.jpg",
      "alt": "",
      "downloaded": true,
      "discovery": "saved_markdown"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=Nxa19uWC_oA",
      "status": "ok",
      "local_path": "attachments/5361b27a4c83/videos/repair-Nxa19uWC_oA.mp4",
      "bytes": 8002774,
      "valid": true,
      "height": 720,
      "width": 1062,
      "duration": "33.361000"
    },
    {
      "url": "https://www.youtube.com/c/TowardsDataScience",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [],
  "retry_at": "2026-10-03T07:58:33",
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:31:50Z",
  "source_url": "https://towardsdatascience.com/neural-networks-intuitively-and-exhaustively-explained-0153f85c1007/",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "http_403",
  "repair_error": "",
  "actual_video_files": [
    {
      "local_path": "attachments/5361b27a4c83/videos/repair-Nxa19uWC_oA.mp4",
      "bytes": 8002774,
      "valid": true,
      "height": 720,
      "width": 1062,
      "duration": "33.361000",
      "status": "ok"
    }
  ],
  "expected_saved_body_images": 61,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# Neural Networks - Intuitively and Exhaustively Explained\nAn in-depth exploration of the most fundamental architecture in modern AI.\n[Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\nFebruary 3, 202525 min read\n### An in-depth exploration of the most fundamental architecture in modern AI\n!["The Thinking Part" by Daniel Warfield using MidJourney. All images by the author unless otherwise specified. Article originally made available on Intuitively and Exhaustively Explained.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1B7C0TLxpFO6fKhqXGRe2bg.png)\n\"The Thinking Part\" by Daniel Warfield using MidJourney. All images by the author unless otherwise specified. Article originally made available on [Intuitively and Exhaustively Explained](https://iaee.substack.com/).\nIn this article we'll form a thorough understanding of the neural network, a cornerstone technology underpinning virtually all cutting edge AI systems. We'll first explore neurons in the human brain, and then explore how they formed the fundamental inspiration for neural networks in AI. We'll then explore back-propagation, the algorithm used to train neural networks to do cool stuff. Finally, after forging a thorough conceptual understanding, we'll implement a Neural Network ourselves from scratch and train it to solve a toy problem.\n* * *\n**Who is this useful for?** Anyone who wants to form a complete understanding of the state of the art of AI.\n**How advanced is this post?** This article is designed to be accessible to beginners, and also contains thorough information which may serve as a useful refresher for more experienced readers.\n**Pre-requisites:** None\n* * *\nLearn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist) roadmap.\n## Inspiration From the Brain\nNeural networks take direct inspiration from the human brain, which is made up of billions of incredibly complex cells called neurons.\n![The Neuron, source](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0H6bVaEAMKw2WlsXB.png)\nThe Neuron, [source](https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png)\nThe process of thinking within the human brain is the result of communication between neurons. You might receive stimulus in the form of something you saw, then that information is propagated to neurons in the brain via electrochemical signals.\n![eye image generated with Midjourney](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xfX_DPDEwe3sUE7N1wIZvw.png)\neye image generated with Midjourney\nThe first neurons in the brain receive that stimulus, then each neuron may choose whether or not to \"fire\" based on how much stimulus it received. \"Firing\", in this case, is a neurons decision to send signals to the neurons it's connected to.\n![Imagine the signal from the eye directly feeds into three neurons, and two decide to fire.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11_-vnr8Ax_TqknfAYA2A-w.png)\nImagine the signal from the eye directly feeds into three neurons, and two decide to fire.\nThen the neurons which those Neurons are connected to may or may not choose to fire.\n![Neurons receive stimulus from previous neurons and then choose whether or not to fire based on the magnitude of the stimulus.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1fIUUxfy2BMODCRmi1D3SBw.png)\nNeurons receive stimulus from previous neurons and then choose whether or not to fire based on the magnitude of the stimulus.\nThus, a \"thought\" can be conceptualized as a large number of neurons choosing to, or not to fire based on the stimulus from other neurons.\nAs one navigates throughout the world, one might have certain thoughts more than another person. A cellist might use some neurons more than a mathematician, for instance.\n![Different tasks require the use of different neurons. Images generated with Midjourney](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v_6qjOhyH3DT0rEj95_zRg.png)\nDifferent tasks require the use of different neurons. Images generated with Midjourney\nWhen we use certain neurons more frequently, their connections become stronger, increasing the intensity of those connections. When we don't use certain neurons, those connections weaken. This general rule has inspired the phrase \"Neurons that fire together, wire together\", and is the high-level quality of the brain which is responsible for the learning process.\n![The process of using certain neurons strengthens their connections.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13opdx0rZVIavS3hd-FtJbw.png)\nThe process of using certain neurons strengthens their connections.\nI'm not a neurologist, so of course this is a tremendously simplified description of the brain. However, it's enough to understand the fundamental idea of a neural network.\n## The Intuition of Neural Networks\nNeural networks are, essentially, a mathematically convenient and simplified version of neurons within the brain. A neural network is made up of elements called \"perceptrons\", which are directly inspired by neurons.\n![A perceptron, on the left, vs a neuron, on the right. \\[source\\]\\(https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png\\) 1, source 2](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Vp3uVTyAcixfAlwcZWTTbQ.png)\nA perceptron, on the left, vs a neuron, on the right. [[source](https://en.wikipedia.org/wiki/Neuron#/media/File:Blausen_0657_MultipolarNeuron.png) 1](https://commons.wikimedia.org/wiki/File:ArtificialNeuronModel_english.png), source 2\nPerceptrons take in data, like a neuron does,\n![Perceptrons in AI work with numbers, while Neurons within the brain work with electrochemical signals.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Ixk7vi9afHRoRx5dG_E37A.png)\nPerceptrons in AI work with numbers, while Neurons within the brain work with electrochemical signals.\naggregate that data, like a neuron does,\n![Perceptrons aggregate numbers to come up with an output, while neurons aggregate electrochemical signals to come up with an output.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1M4Hg_rE7WmLFqpMjfg5P3Q.png)\nPerceptrons aggregate numbers to come up with an output, while neurons aggregate electrochemical signals to come up with an output.\nthen output a signal based on the input, like a neuron does.\n![Perceptrons output numbers, while neurons output electrochemical signals.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1DHnCzMRpcBASTtxFMeAuFg.png)\nPerceptrons output numbers, while neurons output electrochemical signals.\nA neural network can be conceptualized as a big network of these perceptrons, just like the brain is a big network of neurons.\n![A neural network \\(left\\) vs the brain \\(right\\). src1 src2](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1-7K5ccbzd9_UO6UIRzxbOA.png)\nA neural network (left) vs the brain (right). [src1](https://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Artificial_neural_network.svg/560px-Artificial_neural_network.svg.png) [src2](https://www.google.com/url?sa=i&url=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FNeuron&psig=AOvVaw16zt0cwZKdlPi6mQqgRQLT&ust=1736975287779000&source=images&cd=vfe&opi=89978449&ved=0CBQQjRxqFwoTCNi7sKyP9ooDFQAAAAAdAAAAABAE)\nWhen a neuron in the brain fires, it does so as a binary decision. Or, in other words, neurons either fire or they don't. Perceptrons, on the other hand, don't \"fire\" per-se, but output a range of numbers based on the perceptrons input.\n![Perceptrons output a continuous range of numbers, while Neurons either fire or they don't.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11snQ4GLqxc-6tHjXaF6TtA.png)\nPerceptrons output a continuous range of numbers, while Neurons either fire or they don't.\nNeurons within the brain can get away with their relatively simple binary inputs and outputs because thoughts exist over time. Neurons essentially [pulse at different rates](https://www.youtube.com/watch?v=Nxa19uWC_oA), with slower and faster pulses communicating different information.\nSo, neurons have simple inputs and outputs in the form of on or off pulses, but the rate at which they pulse can communicate complex information. Perceptrons only see an input once per pass through the network, but their input and output can be a continuous range of values. If you're familiar with electronics, you might reflect on how this is similar to the relationship between digital and analogue signals.\nThe way the math for a perceptron actually shakes out is pretty simple. A standard neural network consists of a bunch of weights connecting the perceptron's of different layers together.\n![A neural network, with the weights leading into and out of a particular perceptron highlighted.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1cLcbZkQ4KCmyqsoIsjEQCg.png)\nA neural network, with the weights leading into and out of a particular perceptron highlighted.\nYou can calculate the value of a particular perceptron by adding up all the inputs, multiplied by their respective weights.\n![An example of how the value of a perceptron might be calculated. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\)=-0.09](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Sd0t4WYtLZmhvF6w6VcQ6A.png)\nAn example of how the value of a perceptron might be calculated. (0.3×0.3) + (0.7×0.1) +(-0.5×0.5)=-0.09\nMany Neural Networks also have a \"bias\" associated with each perceptron, which is added to the sum of the inputs to calculate the perceptron's value.\n![An example of how the value of a perceptron might be calculated when a bias term is included in the model. \\(0.3×0.3\\) + \\(0.7×0.1\\) +\\(-0.5×0.5\\) + 0.01 =-0.08](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1gTOvMB96VO5qDn-VI7Me_w.png)\nAn example of how the value of a perceptron might be calculated when a bias term is included in the model. (0.3×0.3) + (0.7×0.1) +(-0.5×0.5) + 0.01 =-0.08\nCalculating the output of a neural network, then, is just doing a bunch of addition and multiplication to calculate the value of all the perceptrons.\nSometimes data scientists refer to this general operation as a \"linear projection\", because we're mapping an input into an output via linear operations (addition and multiplication). One problem with this approach is, even if you daisy chain a billion of these layers together, the resulting model will still just be a linear relationship between the input and output because it's all just addition and multiplication.\nThis is a serious problem because not all relationships between an input and output are linear. To get around this, data scientists employ something called an \"activation function\". These are non-linear functions which can be injected throughout the model to, essentially, sprinkle in some non-linearity.\n![Examples of a variety of functions which, given some input, produce some output. The top three are linear, while the bottom three are non-linear.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1q-oox60ZsuYVYbZkp2SpEg.png)\nExamples of a variety of functions which, given some input, produce some output. The top three are linear, while the bottom three are non-linear.\nby interweaving non-linear activation functions between linear projections, neural networks are capable of learning very complex functions,\n![By placing non-linear activation functions within a neural network, neural networks are capable of modeling complex relationships.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1m_PdmmIOUeKQZDM59eehNw.png)\nBy placing non-linear activation functions within a neural network, neural networks are capable of modeling complex relationships.\nIn AI there are many popular activation functions, but the industry has largely converged on three popular ones: ReLU, Sigmoid, and Softmax, which are used in a variety of different applications. Out of all of them, ReLU is the most common due to its simplicity and ability to generalize to mimic almost any other function.\n![The ReLU activation function, where the output is equal to zero if the input is less than zero, and the output is equal to the input if the input is greater than zero.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1U2_yx_M8XCD93RPOxvg9hQ.png)\nThe ReLU activation function, where the output is equal to zero if the input is less than zero, and the output is equal to the input if the input is greater than zero.\nSo, that's the essence of how AI models make predictions. It's a bunch of addition and multiplication with some nonlinear functions sprinkled in between.\nAnother defining characteristic of neural networks is that they can be trained to be better at solving a certain problem, which we'll explore in the next section.\n## Back Propagation\nOne of the fundamental ideas of AI is that you can \"train\" a model. This is done by asking a neural network (which starts its life as a big pile of random data) to do some task. Then, you somehow update the model based on how the model's output compares to a known good answer.\n![The fundamental idea of training a neural network. You give it some data where you know what you want the output to be, compare the neural networks output with your desired result, then use how wrong the neural network was to update the parameters so it's less wrong.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1JYyCqZhkWAvTaZ7fPhnk9A.png)\nThe fundamental idea of training a neural network. You give it some data where you know what you want the output to be, compare the neural networks output with your desired result, then use how wrong the neural network was to update the parameters so it's less wrong.\nFor this section, let's imagine a neural network with an input layer, a hidden layer, and an output layer.\n![A neural network with two inputs and a single output, with a hidden layer in-between allowing the model to make more complex predictions.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1v2Fwu8wCaSoKiTBBpZknuQ.png)\nA neural network with two inputs and a single output, with a hidden layer in-between allowing the model to make more complex predictions.\nEach of these layers are connected together with, initially, completely random weights.\n![The neural network, with random weights and biases defined.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1UD0Z1G4QBuVvud3Sf8gOWg.png)\nThe neural network, with random weights and biases defined.\nAnd we'll use a ReLU activation function on our hidden layer.\n![We'll apply the ReLU activation function to the value of our hidden perceptrons.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1pLo_h8SZrJbAjd42Qeyv_A.png)\nWe'll apply the ReLU activation function to the value of our hidden perceptrons.\nLet's say we have some training data, in which the desired output is the average value of the input.\n![An example of the data that we'll be training off of.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NI1jil4RRhvW8m9yPrWTnQ.png)\nAn example of the data that we'll be training off of.\nAnd we pass an example of our training data through the model, generating a prediction.\n![Calculating the value of the hidden layer and output based on the input, including all major intermediary steps.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15fKc61r8UC33OQqn5v0Gnw.png)\nCalculating the value of the hidden layer and output based on the input, including all major intermediary steps.\nTo make our neural network better at the task of calculating the average of the input, we first compare the predicted output to what our desired output is.\n![The training data has an input of 0.1 and 0.3, and the desired output \\(the average of the input\\) is 0.2. The prediction from the model was -0.1. Thus, the difference between the output and the desired output is 0.3.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NNXP11WMJkbRzxtcEI7eZQ.png)\nThe training data has an input of 0.1 and 0.3, and the desired output (the average of the input) is 0.2. The prediction from the model was -0.1. Thus, the difference between the output and the desired output is 0.3.\nNow that we know that the output should increase in size, we can look back through the model to calculate how our weights and biases might change to promote that change.\nFirst, let's look at the weights leading immediately into the output: w₇, w₈, w₉. Because the output of the third hidden perceptron was -0.46, the activation from ReLU was 0.00.\n![The ultimate, activated output of the third perceptron, 0.00](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1dIbrD12PNayTGNhOD75nmg.png)\nThe ultimate, activated output of the third perceptron, 0.00\nAs a result, there's no change to w₉ that could result us getting closer to our desired output, because every value of w₉ would result in a change of zero in this particular example.\nThe second hidden neuron, however, does have an activated output which is greater than zero, and thus adjusting w₈ will have an impact on the output for this example.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/13Whf1AhkWz6clG06cztpCw.png)\nThe way we actually calculate how much w₈ should change is by multiplying how much the output should change, times the input to w₈.\n![How we calculate how the weight should change. Here the symbol Δ\\(delta\\) means "change in", so Δw₈ means the "change in w₈"](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1aT1An9_PHK2UdYPg4P396g.png)\nHow we calculate how the weight should change. Here the symbol Δ(delta) means \"change in\", so Δw₈ means the \"change in w₈\"\nThe easiest explanation of why we do it this way is \"because calculus\", but if we look at how all weights get updated in the last layer, we can form a fun intuition.\n![Calculating how the weights leading into the output should change.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1kmHCi-J7Fg3IV6aEynIagQ.png)\nCalculating how the weights leading into the output should change.\nNotice how the two perceptrons that \"fire\" (have an output greater than zero) are updated together. Also, notice how the stronger a perceptrons output is, the more its corresponding weight is updated. This is somewhat similar to the idea that \"Neurons that fire together, wire together\" within the human brain.\nCalculating the change to the output bias is super easy. In fact, we've already done it. Because the bias is how much a perceptrons output should change, the change in the bias is just the changed in the desired output. So, Δb₄=0.3\n![how the bias of the output should be updated.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1yU1vGFGDcYF_bsYpi1EViQ.png)\nhow the bias of the output should be updated.\nNow that we've calculated how the weights and bias of the output perceptron should change, we can \"back propagate\" our desired change in output through the model. Let's start with back propagating so we can calculate how we should update w₁.\nFirst, we calculate how the activated output of the of the first hidden neuron should change. We do that by multiplying the change in output by w₇.\n![Calculating how the activated output of the first hidden neuron should have changed by multiplying the desired change in the output by w₇.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15bZbQeeglMZ8eNSAMBADJg.png)\nCalculating how the activated output of the first hidden neuron should have changed by multiplying the desired change in the output by w₇.\nFor values that are greater than zero, ReLU simply multiplies those values by 1. So, for this example, the change we want the un-activated value of the first hidden neuron is equal to the desired change in the activated output\n![How much we want to change the un-activated value of the first hidden perceptron, based on back-propagating from the output.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1MG5zTdstAdlxDYriLA1dMw.png)\nHow much we want to change the un-activated value of the first hidden perceptron, based on back-propagating from the output.\nRecall that we calculated how to update w₇ based on multiplying it's input by the change in its desired output. We can do the same thing to calculate the change in w₁.\n![Now that we've calculated how the first hidden neuron should change, we can calculate how we should update w₁ the same way we calculated how w₇ should be updated previously.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1rb2kMoJK5HkgeBIvWeXYZA.png)\nNow that we've calculated how the first hidden neuron should change, we can calculate how we should update w₁ the same way we calculated how w₇ should be updated previously.\nIt's important to note, we're not actually updating any of the weights or biases throughout this process. Rather, we're taking a tally of how we should update each parameter, assuming no other parameters are updated.\nSo, we can do those calculations to calculate all parameter changes.\n![By back propagating through the model, using a combination of values from the forward passes and desired changes from the backward pass at various points of the model, we can calculate how all parameters should change](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/15zQm78JT5IjAvQSI0YdEdw.png)\nBy back propagating through the model, using a combination of values from the forward passes and desired changes from the backward pass at various points of the model, we can calculate how all parameters should change\nA fundamental idea of back propagation is called \"Learning Rate\", which concerns the size of the changes we make to neural networks based on a particular batch of data. To explain why this is important, I'd like to use an analogy.\nImagine you went outside one day, and everyone wearing a hat gave you a funny look. You probably don't want to jump to the conclusion that `wearing hat = funny look` , but you might be a bit skeptical of people wearing hats. After three, four, five days, a month, or even a year, if it seems like the vast majority of people wearing hats are giving you a funny look, you may start considering that a strong trend.\nSimilarly, when we train a neural network, we don't want to completely change how the neural network thinks based on a single training example. Rather, we want each batch to only incrementally change how the model thinks. As we expose the model to many examples, we would hope that the model would learn important trends within the data.\nAfter we've calculated how each parameter should change as if it were the only parameter being updated, we can multiply all those changes by a small number, like `0.001` , before applying those changes to the parameters. This small number is commonly referred to as the \"learning rate\", and the exact value it should have is dependent on the model we're training on. This effectively scales down our adjustments before applying them to the model.\nAt this point we covered pretty much everything one would need to know to implement a neural network. Let's give it a shot!\n![Join IAEE](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0SOOewKyrD86bJlSQ.jpeg)\n[Join IAEE](https://iaee.substack.com/)\n## Implementing a Neural Network from Scratch\nTypically, a data scientist would just use a library like `PyTorch` to implement a neural network in a few lines of code, but we'll be defining a neural network from the ground up using NumPy, a numerical computing library.\nFirst, let's start with a way to define the structure of the neural network.\npython\n\n```\n\"\"\"Blocking out the structure of the Neural Network\n\"\"\"\n\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\nprint('weight dimensions:')\nfor w in model.weights:\n    print(w.shape)\n\nprint('nbias dimensions:')\nfor b in model.biases:\n    print(b.shape)\n```\n\n![The weight and bias matrix defined in a sample neural network.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1Bn8-3lou0sBs8y2cf1sY8g.png)\nThe weight and bias matrix defined in a sample neural network.\nWhile we typically draw neural networks as a dense web in reality we represent the weights between their connections as matrices. This is convenient because matrix multiplication, then, is equivalent to passing data through a neural network.\n![Thinking of a dense network as weighted connections on the left, and as matrix multiplication on the right. On the right hand side diagram, the vector on the left would be the input, the matrix in the center would be the weight matrix, and the vector on the right would be the output. Only a portion of values are included for readability. From my article on LoRA.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/02kqALjWcLyiGL9YQ.png)\nThinking of a dense network as weighted connections on the left, and as matrix multiplication on the right. On the right hand side diagram, the vector on the left would be the input, the matrix in the center would be the weight matrix, and the vector on the right would be the output. Only a portion of values are included for readability. From my article on [LoRA](https://medium.com/towards-data-science/lora-intuitively-and-exhaustively-explained-e944a6bff46b).\nWe can make our model make a prediction based on some input by passing the input through each layer.\npython\n\n```\n\"\"\"Implementing the Forward Pass\n\"\"\"\n\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        #implementing the relu activation function\n        return np.maximum(0, x)\n\n    def forward(self, X):\n        #iterating through all layers\n        for W, b in zip(self.weights, self.biases):\n\n            #applying the weight and bias of the layer\n            X = np.dot(X, W) + b\n\n            #doing ReLU for all but the last layer\n            if W is not self.weights[-1]:\n                X = self.relu(X)\n\n        #returning the result\n        return X\n\n    def predict(self, X):\n        y = self.forward(X)\n        return y.flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Generate predictions\nprediction = model.predict(np.array([0.1,0.2]))\nprint(prediction)\n```\n\n![the result of passing our data through the model. Our model is randomly defined, so this isn't a useful prediction, but it confirms that the model is working.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/12gnRkuco9SpHeEpKaynhrQ.png)\nthe result of passing our data through the model. Our model is randomly defined, so this isn't a useful prediction, but it confirms that the model is working.\nWe need to be able to train this model, and to do that we'll first need a problem to train the model on. I defined a random function that takes in two inputs and results in an output:\npython\n\n```\n\"\"\"Defining what we want the model to learn\n\"\"\"\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n# Define a random function with two inputs\ndef random_function(x, y):\n    return (np.sin(x) + x * np.cos(y) + y + 3**(x/3))\n\n# Generate a grid of x and y values\nx = np.linspace(-10, 10, 100)\ny = np.linspace(-10, 10, 100)\nX, Y = np.meshgrid(x, y)\n\n# Compute the output of the random function\nZ = random_function(X, Y)\n\n# Create a 2D plot\nplt.figure(figsize=(8, 6))\ncontour = plt.contourf(X, Y, Z, cmap='viridis')\nplt.colorbar(contour, label='Function Value')\nplt.title('2D Plot of Objective Function')\nplt.xlabel('X-axis')\nplt.ylabel('Y-axis')\nplt.show()\n```\n\n![The modeling objective. Given two inputs \\(here plotted as x and y\\), the model needs to predict an output \\(here represented as color\\). This is a completely arbitrary function](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11f28YdNHVOI7goP3REmrfA.png)\nThe modeling objective. Given two inputs (here plotted as x and y), the model needs to predict an output (here represented as color). This is a completely arbitrary function\nIn the real world we wouldn't know the underlying function. We can mimic that reality by creating a dataset consisting of random points:\npython\n\n```\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\n# Define a random function with two inputs\ndef random_function(x, y):\n    return (np.sin(x) + x * np.cos(y) + y + 3**(x/3))\n\n# Define the number of random samples to generate\nn_samples = 1000\n\n# Generate random X and Y values within a specified range\nx_min, x_max = -10, 10\ny_min, y_max = -10, 10\n\n# Generate random values for X and Y\nX_random = np.random.uniform(x_min, x_max, n_samples)\nY_random = np.random.uniform(y_min, y_max, n_samples)\n\n# Evaluate the random function at the generated X and Y values\nZ_random = random_function(X_random, Y_random)\n\n# Create a dataset\ndataset = pd.DataFrame({\n    'X': X_random,\n    'Y': Y_random,\n    'Z': Z_random\n})\n\n# Display the dataset\nprint(dataset.head())\n\n# Create a 2D scatter plot of the sampled data\nplt.figure(figsize=(8, 6))\nscatter = plt.scatter(dataset['X'], dataset['Y'], c=dataset['Z'], cmap='viridis', s=10)\nplt.colorbar(scatter, label='Function Value')\nplt.title('Scatter Plot of Randomly Sampled Data')\nplt.xlabel('X-axis')\nplt.ylabel('Y-axis')\nplt.show()\n```\n\n![This is the data we'll be training on to try to learn our function.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nw8UuI7bSKM_Qth-LYCeMA.png)\nThis is the data we'll be training on to try to learn our function.\nRecall that the back propagation algorithm updates parameters based on what happens in a forward pass. So, before we implement backpropagation itself, let's keep track of a few important values in the forward pass: The inputs and outputs of each perceptron throughout the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        #keeping track of these values in this code block\n        #so we can observe them\n        self.perceptron_inputs = None\n        self.perceptron_outputs = None\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    def forward(self, X):\n        self.perceptron_inputs = [X]\n        self.perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(self.perceptron_inputs[-1], W) + b\n            self.perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            self.perceptron_inputs.append(A)\n\n        return self.perceptron_inputs, self.perceptron_outputs\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Generate predictions\nprediction = model.predict(np.array([0.1,0.2]))\n\n#looking through critical optimization values\nfor i, (inpt, outpt) in enumerate(zip(model.perceptron_inputs, model.perceptron_outputs[:-1])):\n    print(f'layer {i}')\n    print(f'input: {inpt.shape}')\n    print(f'output: {outpt.shape}')\n    print('')\n\nprint('Final Output:')\nprint(model.perceptron_outputs[-1].shape)\n```\n\n![The values throughout various layers of the model as a result of the forward pass. This will allow us to compute the necessary changes to update the model.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/17hmH0UrcyzSjmfLAABusZA.png)\nThe values throughout various layers of the model as a result of the forward pass. This will allow us to compute the necessary changes to update the model.\nNow that we have a record stored of critical intermediary value within the network, we can use those values, along with the error of a model for a particular prediction, to calculate the changes we should make to the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    @staticmethod\n    def relu_as_weights(x):\n        return (x > 0).astype(float)\n\n    def forward(self, X):\n        perceptron_inputs = [X]\n        perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(perceptron_inputs[-1], W) + b\n            perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            perceptron_inputs.append(A)\n\n        return perceptron_inputs, perceptron_outputs\n\n    def backward(self, perceptron_inputs, perceptron_outputs, target):\n        weight_changes = []\n        bias_changes = []\n\n        m = len(target)\n        dA = perceptron_inputs[-1] - target.reshape(-1, 1)  # Output layer gradient\n\n        for i in reversed(range(len(self.weights))):\n            dZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n            dW = np.dot(perceptron_inputs[i].T, dZ) / m\n            db = np.sum(dZ, axis=0, keepdims=True) / m\n            weight_changes.append(dW)\n            bias_changes.append(db)\n\n            if i > 0:\n                dA = np.dot(dZ, self.weights[i].T)\n\n        return list(reversed(weight_changes)), list(reversed(bias_changes))\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n\n#defining a model\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n#defining a sample input and target output\ninput = np.array([[0.1,0.2]])\ndesired_output = np.array([0.5])\n\n#doing forward and backward pass to calculate changes\nperceptron_inputs, perceptron_outputs = model.forward(input)\nweight_changes, bias_changes = model.backward(perceptron_inputs, perceptron_outputs, desired_output)\n\n#smaller numbers for printing\nnp.set_printoptions(precision=2)\n\nfor i, (layer_weights, layer_biases, layer_weight_changes, layer_bias_changes)\nin enumerate(zip(model.weights, model.biases, weight_changes, bias_changes)):\n    print(f'layer {i}')\n    print(f'weight matrix: {layer_weights.shape}')\n    print(f'weight matrix changes: {layer_weight_changes.shape}')\n    print(f'bias matrix: {layer_biases.shape}')\n    print(f'bias matrix changes: {layer_bias_changes.shape}')\n    print('')\n\nprint('The weight and weight change matrix of the second layer:')\nprint('weight matrix:')\nprint(model.weights[1])\nprint('change matrix:')\nprint(weight_changes[1])\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1J7TFKgd0gBikRMuiMin2Rw.png)\nThis is probably the most complex implementation step, so I want to take a moment to dig through some of the details. The fundamental idea is exactly as we described in previous sections. We're iterating over all layers, from back to front, and calculating what change to each weight and bias would result in a better output.\npython\n\n```\n# calculating output error\ndA = perceptron_inputs[-1] - target.reshape(-1, 1)\n\n#a scaling factor for the batch size.\n#you want changes to be an average across all batches\n#so we divide by m once we've aggregated all changes.\nm = len(target)\n\nfor i in reversed(range(len(self.weights))):\n  dZ = dA #simplified for now\n\n  # calculating change to weights\n  dW = np.dot(perceptron_inputs[i].T, dZ) / m\n  # calculating change to bias\n  db = np.sum(dZ, axis=0, keepdims=True) / m\n\n  # keeping track of required changes\n  weight_changes.append(dW)\n  bias_changes.append(db)\n  ...\n```\n\nCalculating the change to bias is pretty straight forward. If you look at how the output of a given neuron should have impacted all future neurons, you can add up all those values (which are both positive and negative) to get an idea of if the neuron should be biased in a positive or negative direction.\nThe way we calculate the change to weights, by using matrix multiplication, is a bit more mathematically complex.\nini\n\n```\ndW = np.dot(perceptron_inputs[i].T, dZ) / m\n```\n\nBasically, this line says that the change in the weight should be equal to the value going into the perceptron, times how much the output should have changed. If a perceptron had a big input, the change to its outgoing weights should be a large magnitude, if the perceptron had a small input, the change to its outgoing weights will be small. Also, if a weight points towards an output which should change a lot, the weight should change a lot.\nThere is another line we should discuss in our back propagation implement.\nlua\n\n```\ndZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n```\n\nIn this particular network, there are activation functions throughout the network, following all but the final output. When we do back propagation, we need to back-propagate through these activation functions so that we can update the neurons which lie before them. We do this for all but the last layer, which doesn't have an activation function, which is why `dZ = dA if i == len(self.weights) - 1` .\nIn fancy math speak we would call this a derivative, but because I don't want to get into calculus, I called the function `relu_as_weights` . Basically, we can treat each of our ReLU activations as something like a tiny neural network, who's weight is a function of the input. If the input of the ReLU activation function is less than zero, then that's like passing that input through a neural network with a weight of zero. If the input of ReLU is greater than zero, then that's like passing the input through a neural netowork with a weight of one.\n![Recall the ReLU activation function.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1U2_yx_M8XCD93RPOxvg9hQ.png)\nRecall the ReLU activation function.\nThis is exactly what the `relu_as_weights` function does.\npython\n\n```\ndef relu_as_weights(x):\n        return (x > 0).astype(float)\n```\n\nUsing this logic we can treat back propagating through ReLU just like we back propagate through the rest of the neural network.\nAgain, I'll be covering this concept from a more robust mathematical prospective soon, but that's the essential idea from a conceptual perspective.\nNow that we have the forward and backward pass implemented, we can implement training the model.\npython\n\n```\nimport numpy as np\n\nclass SimpleNN:\n    def __init__(self, architecture):\n        self.architecture = architecture\n        self.weights = []\n        self.biases = []\n\n        # Initialize weights and biases\n        np.random.seed(99)\n        for i in range(len(architecture) - 1):\n            self.weights.append(np.random.uniform(\n                low=-1, high=1,\n                size=(architecture[i], architecture[i+1])\n            ))\n            self.biases.append(np.zeros((1, architecture[i+1])))\n\n    @staticmethod\n    def relu(x):\n        return np.maximum(0, x)\n\n    @staticmethod\n    def relu_as_weights(x):\n        return (x > 0).astype(float)\n\n    def forward(self, X):\n        perceptron_inputs = [X]\n        perceptron_outputs = []\n\n        for W, b in zip(self.weights, self.biases):\n            Z = np.dot(perceptron_inputs[-1], W) + b\n            perceptron_outputs.append(Z)\n\n            if W is self.weights[-1]:  # Last layer (output)\n                A = Z  # Linear output for regression\n            else:\n                A = self.relu(Z)\n            perceptron_inputs.append(A)\n\n        return perceptron_inputs, perceptron_outputs\n\n    def backward(self, perceptron_inputs, perceptron_outputs, y_true):\n        weight_changes = []\n        bias_changes = []\n\n        m = len(y_true)\n        dA = perceptron_inputs[-1] - y_true.reshape(-1, 1)  # Output layer gradient\n\n        for i in reversed(range(len(self.weights))):\n            dZ = dA if i == len(self.weights) - 1 else dA * self.relu_as_weights(perceptron_outputs[i])\n            dW = np.dot(perceptron_inputs[i].T, dZ) / m\n            db = np.sum(dZ, axis=0, keepdims=True) / m\n            weight_changes.append(dW)\n            bias_changes.append(db)\n\n            if i > 0:\n                dA = np.dot(dZ, self.weights[i].T)\n\n        return list(reversed(weight_changes)), list(reversed(bias_changes))\n\n    def update_weights(self, weight_changes, bias_changes, lr):\n        for i in range(len(self.weights)):\n            self.weights[i] -= lr * weight_changes[i]\n            self.biases[i] -= lr * bias_changes[i]\n\n    def train(self, X, y, epochs, lr=0.01):\n        for epoch in range(epochs):\n            perceptron_inputs, perceptron_outputs = self.forward(X)\n            weight_changes, bias_changes = self.backward(perceptron_inputs, perceptron_outputs, y)\n            self.update_weights(weight_changes, bias_changes, lr)\n\n            if epoch % 20 == 0 or epoch == epochs - 1:\n                loss = np.mean((perceptron_inputs[-1].flatten() - y) ** 2)  # MSE\n                print(f\"EPOCH {epoch}: Loss = {loss:.4f}\")\n\n    def predict(self, X):\n        perceptron_inputs, _ = self.forward(X)\n        return perceptron_inputs[-1].flatten()\n```\n\nThe `train` function:\n  * iterates through all the data some number of times (defined by `epoch` )\n  * passes the data through a forward pass\n  * calculates how the weights and biases should change\n  * updates the weights and biases, by scaling their changes by the learning rate ( `lr` )\n\n\nAnd thus we've implemented a neural network! Let's train it.\n## Training and Evaluating the Neural Network.\nRecall that we defined an arbitrary 2D function we wanted to learn how to emulate,\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/11f28YdNHVOI7goP3REmrfA.png)\nand we sampled that space with some number of points, which we're using to train the model.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1NESTsjN_9dOKVmWgP12ZGA.png)\nBefore feeding this data into our model, it's vital that we first \"normalize\" the data. Certain values of the dataset are very small or very large, which can make training a neural network very difficult. Values within the neural network can quickly grow to absurdly large values, or diminish to zero, which can inhibit training. Normalization squashes all of our inputs, and our desired outputs, into a more reasonable range averaging around zero with a standardized distribution called a \"normal\" distribution.\nini\n\n```\n# Flatten the data\nX_flat = X.flatten()\nY_flat = Y.flatten()\nZ_flat = Z.flatten()\n\n# Stack X and Y as input features\ninputs = np.column_stack((X_flat, Y_flat))\noutputs = Z_flat\n\n# Normalize the inputs and outputs\ninputs_mean = np.mean(inputs, axis=0)\ninputs_std = np.std(inputs, axis=0)\noutputs_mean = np.mean(outputs)\noutputs_std = np.std(outputs)\n\ninputs = (inputs - inputs_mean) / inputs_std\noutputs = (outputs - outputs_mean) / outputs_std\n```\n\nIf we want to get back predictions in the actual range of data from our original dataset, we can use these values to essentially \"un-squash\" the data.\nOnce we've done that, we can define and train our model.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=2000, lr=0.001)\n```\n\n![As can be seen, the value of loss is going down consistently, implying the model is improving.](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1nXquA-yHCZrfdwSHSWwvzw.png)\nAs can be seen, the value of loss is going down consistently, implying the model is improving.\nThen we can visualize the output of the neural network's prediction vs the actual function.\npython\n\n```\nimport matplotlib.pyplot as plt\n\n# Reshape predictions to grid format for visualization\nZ_pred = model.predict(inputs) * outputs_std + outputs_mean\nZ_pred = Z_pred.reshape(X.shape)\n\n# Plot comparison of the true function and the model predictions\nfig, axes = plt.subplots(1, 2, figsize=(14, 6))\n\n# Plot the true function\naxes[0].contourf(X, Y, Z, cmap='viridis')\naxes[0].set_title(\"True Function\")\naxes[0].set_xlabel(\"X-axis\")\naxes[0].set_ylabel(\"Y-axis\")\naxes[0].colorbar = plt.colorbar(axes[0].contourf(X, Y, Z, cmap='viridis'), ax=axes[0], label=\"Function Value\")\n\n# Plot the predicted function\naxes[1].contourf(X, Y, Z_pred, cmap='plasma')\naxes[1].set_title(\"NN Predicted Function\")\naxes[1].set_xlabel(\"X-axis\")\naxes[1].set_ylabel(\"Y-axis\")\naxes[1].colorbar = plt.colorbar(axes[1].contourf(X, Y, Z_pred, cmap='plasma'), ax=axes[1], label=\"Function Value\")\n\nplt.tight_layout()\nplt.show()\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1WD5ABUHvbEoC2Bg8Z-Ob6w.png)\nThis did an ok job, but not as great as we might like. This is where a lot of data scientists spend their time, and there are a ton of approaches to making a neural network fit a certain problem better. Some obvious ones are:\n  * use more data\n  * play around with the learning rate\n  * train for more epochs\n  * change the structure of the model\n\n\nIt's pretty easy for us to crank up the amount of data we're training on. Let's see where that leads us. Here I'm sampling our dataset 10,000 times, which is 10x more training samples than our previous dataset.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1V9NO7sBco4qsMgpzPU6e9w.png)\nAnd then I trained the model just like before, except this time it took a lot longer because each epoch now analyses 10,000 samples rather than 1,000.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=2000, lr=0.001)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/14RK1Zw7_QvVMncehuB5aLg.png)\nI then rendered the output of this model, the same way I did before, but it didn't really look like the output got much better.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1xi2uWp2nG_RW745IEOplPw.png)\nLooking back at the loss output from training, it seems like the loss is still steadily declining. Maybe I just need to train for longer. Let's try that.\nmakefile\n\n```\n# Define the architecture: [input_dim, hidden1, ..., output_dim]\narchitecture = [2, 64, 64, 64, 1]  # Two inputs, two hidden layers, one output\nmodel = SimpleNN(architecture)\n\n# Train the model\nmodel.train(inputs, outputs, epochs=4000, lr=0.001)\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1LuD6WcQFUxI3-kLmOxZUkw.png)\nThe results seem to be a bit better, but they aren't' amazing.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/1QOcHdFMieEoEKvOdPjzMzg.png)\nI'll spare you the details. I ran this a few times, and I got some decent results, but never anything 1 to 1. I'll be covering some more advanced approaches data scientists use, like annealing and dropout, in future articles which will result in a more consistent and better output. Still, though, we made a neural network from scratch and trained it to do something, and it did a decent job! Pretty neat!\n## Conclusion\nIn this article we avoided calculus like the plague while simultaneously forging an understanding of Neural Networks. We explored their theory, a little bit about the math, the idea of back propagation, and then implemented a neural network from scratch. We then applied a neural network to a toy problem, and explored some of the simple ideas data scientists employ to actually train neural networks to be good at things.\nIn future articles we'll explore a few more advanced approaches to Neural Networks, so stay tuned! For now, you might be interested in a more thorough analysis of Gradients, the fundamental math behind back propagation.\n> [**What Are Gradients, and Why Do They Explode?**](https://towardsdatascience.com/what-are-gradients-and-why-do-they-explode-add23264d24b)\nYou might also be interested in this article, which covers training a neural network using more conventional data science tools like PyTorch.\n> [**AI for the Absolute Novice - Intuitively and Exhaustively Explained**](https://towardsdatascience.com/ai-for-the-absolute-novice-intuitively-and-exhaustively-explained-7b353a31e6d7)\n## Join Intuitively and Exhaustively Explained\nAt IAEE you can find:\n  * Long form content, like the article you just read\n  * Conceptual breakdowns of some of the most cutting-edge AI topics\n  * By-Hand walkthroughs of critical mathematical operations in AI\n  * Practical tutorials and explainers\n\n\n![Join IAEE](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/02/0tBoYEtqCyiht-7gI.jpeg)\n[Join IAEE](https://iaee.substack.com/)\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657187493_78jlbn.jpeg)\nWritten By\n## [Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\n[See all from Daniel Warfield](https://towardsdatascience.com/author/danielwarfield1/)\n[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[AI](https://towardsdatascience.com/tag/ai/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Neural Networks](https://towardsdatascience.com/tag/neural-networks/)[Software Development](https://towardsdatascience.com/tag/software-development/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&title=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&title=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fneural-networks-intuitively-and-exhaustively-explained-0153f85c1007%2F&text=Neural%20Networks%20-%20Intuitively%20and%20Exhaustively%20Explained)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/01/0ZOz1JOTOg2dKz00B-scaled.jpg)](https://towardsdatascience.com/choosing-the-right-tool-for-your-next-data-science-web-application-3d7ec8293d61/)\n## [Choosing the Right Tool for Your Next Data Science Web Application](https://towardsdatascience.com/choosing-the-right-tool-for-your-next-data-science-web-application-3d7ec8293d61/)\n[Programming](https://towardsdatascience.com/category/programming/)\nFlask, Django, Streamlit: Three options for data scientists getting into web development.\n[Murtaza Ali](https://towardsdatascience.com/author/murtaza5152-ali/)\nJanuary 31, 20237 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/09/0wuc6dhYpBqr8YvNn.png)](https://towardsdatascience.com/towards-removing-gender-bias-in-writing-f1307d0a2b71/)\n## [Towards Removing Gender Bias in Writing](https://towardsdatascience.com/towards-removing-gender-bias-in-writing-f1307d0a2b71/)\n[NLP](https://towardsdatascience.com/category/artificial-intelligence/nlp/)\nLearn how to understand the amount of gender bias in your daily consumption of literature and avoid it in your own writing.\n[Srihitha Pallapothula](https://towardsdatascience.com/author/spallapothula2004/)\nSeptember 30, 202111 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/04/1sS20G4rBJLNNwCzBs0g7gA.png)](https://towardsdatascience.com/normalization-vs-standardization-which-one-is-better-f29e043a57eb/)\n## [Normalization vs Standardization, which one is better](https://towardsdatascience.com/normalization-vs-standardization-which-one-is-better-f29e043a57eb/)\n[Programming](https://towardsdatascience.com/category/programming/)\nIn this tutorial let us see which one is the best feature engineering technique of them all.\n[Tanu N Prabhu](https://towardsdatascience.com/author/tanunprabhu95/)\nApril 22, 20205 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/01/05XVjt3c2YulvMs43-scaled.jpg)](https://towardsdatascience.com/how-to-improve-ar-predictions-using-arma-for-time-series-analysis-483b84984490/)\n## [How To Improve AR Predictions Using ARMA For Time Series Analysis](https://towardsdatascience.com/how-to-improve-ar-predictions-using-arma-for-time-series-analysis-483b84984490/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nLearn to use an ARMA model to make better predictions than an AR model.\n[Jacob Toftgaard Rasmussen](https://towardsdatascience.com/author/jacobtoftgaardrasmussen/)\nJanuary 14, 20225 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/03/1CnM4oJOQBSt33oLgLSLMog.png)](https://towardsdatascience.com/causal-machine-learning-what-can-we-accomplish-with-a-single-theorem-ea81432f1fac/)\n## [Causal Machine Learning: What Can We Accomplish with a Single Theorem?](https://towardsdatascience.com/causal-machine-learning-what-can-we-accomplish-with-a-single-theorem-ea81432f1fac/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nExploring and exploiting the seemingly innocent theorem behind Double Machine Learning\n[Harrison Hoffman](https://towardsdatascience.com/author/harrisonfhoffman/)\nMarch 30, 202429 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/07/1ko3qDrEmcYHOwvdvj0wBnA-scaled.jpeg)](https://towardsdatascience.com/the-unknown-features-of-pythons-operator-module-1ad9075d9536/)\n## [The Unknown Features of Python's Operator Module](https://towardsdatascience.com/the-unknown-features-of-pythons-operator-module-1ad9075d9536/)\n[Programming](https://towardsdatascience.com/category/programming/)\nMake your code faster, concise, more readable and more functional with help of Python's little known operator module\n[Martin Heinz](https://towardsdatascience.com/author/martin.heinz/)\nJuly 26, 20216 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/0-DMUzqsJxapVIqS9-scaled.jpg)](https://towardsdatascience.com/stop-being-data-driven-583b5e7abe7b/)\n## [Stop Being Data-Driven](https://towardsdatascience.com/stop-being-data-driven-583b5e7abe7b/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nWhy we are fooled by data and how to stop it\n[Cai Parry-Jones](https://towardsdatascience.com/author/caiparryjones96/)\nAugust 29, 202411 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2018/02/1i18hfdYba3iEiY6MCSsiAw.jpeg)](https://towardsdatascience.com/ai-prototypes-for-clients-as-a-marketing-strategy-70de404d27b5/)\n## [AI Prototypes For Clients As A Marketing Strategy](https://towardsdatascience.com/ai-prototypes-for-clients-as-a-marketing-strategy-70de404d27b5/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWhere is the line? Isn't the goal to get paid?\n[Mathieu Lemay](https://towardsdatascience.com/author/lsci/)\nFebruary 14, 20185 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/09/0jMeWfwsZhRDhecGB-scaled.jpg)](https://towardsdatascience.com/how-to-test-pandas-etl-data-pipeline-e49fb5dac4ce/)\n## [How to Test Pandas ETL Data Pipeline](https://towardsdatascience.com/how-to-test-pandas-etl-data-pipeline-e49fb5dac4ce/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nTest your Pandas ETL data pipeline with Great Expectations\n[Edwin Tan](https://towardsdatascience.com/author/edwin.tan/)\nSeptember 19, 20228 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\nword word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word\n  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1  \nmmMwWLliI0fiflO&1\n####  Recommended Articles \nClose\n",
  "body_base_path": "attachments/5361b27a4c83/pages/",
  "body_markdown_path": "attachments/5361b27a4c83/pages/archived-body.md",
  "local_html": "attachments/5361b27a4c83/pages/archived-body.html",
  "body_source": "prior_capture"
}