{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Vuk Rosić",
  "title_listed": "Code a Neural Network from Scratch in NumPy",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/vuk_rosic/code-a-neural-network-from-scratch-in-numpy-494o",
  "publish_date": "2025-07-07",
  "corpus_stage": "",
  "notes": "",
  "id": "e89b1bae3e6e",
  "collected_at": "2026-10-02T14:06:36",
  "title": "Code a Neural Network from Scratch in NumPy - DEV Community",
  "body_markdown": "Part of my\nZero to AI Researcher / Engineer Course\nPart 1: Getting Started - Your First Neural Network\nimport\nnumpy\nas\nnp\nimport\nmatplotlib.pyplot\nas\nplt\n# Create simple dataset - XOR problem\nX\n=\nnp\n.\narray\n([[\n0\n,\n0\n],\n[\n0\n,\n1\n],\n[\n1\n,\n0\n],\n[\n1\n,\n1\n]])\ny\n=\nnp\n.\narray\n([[\n0\n],\n[\n1\n],\n[\n1\n],\n[\n0\n]])\nprint\n(\nf\n\"\nInput shape:\n{\nX\n.\nshape\n}\n\"\n)\n# (4, 2)\nprint\n(\nf\n\"\nOutput shape:\n{\ny\n.\nshape\n}\n\"\n)\n# (4, 1)\nprint\n(\nf\n\"\nDataset:\n\"\n)\nfor\ni\nin\nrange\n(\nlen\n(\nX\n)):\nprint\n(\nf\n\"\n{\nX\n[\ni\n]\n}\n->\n{\ny\n[\ni\n][\n0\n]\n}\n\"\n)\n# Print each input pair and its expected output\nEnter fullscreen mode\nExit fullscreen mode\nWhat happened\n: We created the XOR dataset - a classic non-linearly separable problem that requires a neural network to solve.\nPart 2: Understanding the Math - Forward Pass\nBasic Network Architecture\n# Network architecture: 2 -> 4 -> 1 (input -> hidden -> output)\ninput_size\n=\n2\n# Number of input features (x and y coordinates)\nhidden_size\n=\n4\n# Number of neurons in hidden layer\noutput_size\n=\n1\n# Number of output values (0 or 1)\n# Initialize weights and biases\nnp\n.\nrandom\n.\nseed\n(\n42\n)\nW1\n=\nnp\n.\nrandom\n.\nrandn\n(\ninput_size\n,\nhidden_size\n)\n*\n0.5\n# Weights: connection strengths between layers\nb1\n=\nnp\n.\nzeros\n((\n1\n,\nhidden_size\n))\n# Biases: adjustable offsets for each neuron\nW2\n=\nnp\n.\nrandom\n.\nrandn\n(\nhidden_size\n,\noutput_size\n)\n*\n0.5\n# Weights for output layer\nb2\n=\nnp\n.\nzeros\n((\n1\n,\noutput_size\n))\n# Bias for output layer\nprint\n(\nf\n\"\nW1 shape:\n{\nW1\n.\nshape\n}\n\"\n)\n# (2, 4)\nprint\n(\nf\n\"\nb1 shape:\n{\nb1\n.\nshape\n}\n\"\n)\n# (1, 4)\nprint\n(\nf\n\"\nW2 shape:\n{\nW2\n.\nshape\n}\n\"\n)\n# (4, 1)\nprint\n(\nf\n\"\nb2 shape:\n{\nb2\n.\nshape\n}\n\"\n)\n# (1, 1)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Matrix Shapes in Neural Networks\n# Let's trace through the shapes step by step\nprint\n(\n\"\nShape flow through network:\n\"\n)\nprint\n(\nf\n\"\nInput X:\n{\nX\n.\nshape\n}\n\"\n)\n# (4, 2)\nprint\n(\nf\n\"\nWeights W1:\n{\nW1\n.\nshape\n}\n\"\n)\n# (2, 4)\nprint\n(\nf\n\"\nX @ W1:\n{\n(\nX\n@\nW1\n).\nshape\n}\n\"\n)\n# (4, 4)\nprint\n(\nf\n\"\nBias b1:\n{\nb1\n.\nshape\n}\n\"\n)\n# (1, 4)\n# Broadcasting explanation\nsample_mult\n=\nX\n@\nW1\nprint\n(\nf\n\"\n\\n\nBroadcasting bias:\n\"\n)\nprint\n(\nf\n\"\n(X @ W1) shape:\n{\nsample_mult\n.\nshape\n}\n\"\n)\n# (4, 4)\nprint\n(\nf\n\"\nb1 shape:\n{\nb1\n.\nshape\n}\n\"\n)\n# (1, 4)\nprint\n(\nf\n\"\nResult shape:\n{\n(\nsample_mult\n+\nb1\n).\nshape\n}\n\"\n)\n# (4, 4)\nEnter fullscreen mode\nExit fullscreen mode\nForward Pass Implementation\ndef\nforward_pass\n(\nX\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n):\n\"\"\"\nComplete forward pass through the network\n\"\"\"\n# Hidden layer\nz1\n=\nX\n@\nW1\n+\nb1\n# Linear transformation\na1\n=\nsigmoid\n(\nz1\n)\n# Activation\n# Output layer\nz2\n=\na1\n@\nW2\n+\nb2\n# Linear transformation\na2\n=\nsigmoid\n(\nz2\n)\n# Activation\nreturn\nz1\n,\na1\n,\nz2\n,\na2\n# Test forward pass\nz1\n,\na1\n,\nz2\n,\npredictions\n=\nforward_pass\n(\nX\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n)\nprint\n(\nf\n\"\nPredictions shape:\n{\npredictions\n.\nshape\n}\n\"\n)\nprint\n(\nf\n\"\nPredictions:\n\\n\n{\npredictions\n.\nflatten\n()\n}\n\"\n)\nprint\n(\nf\n\"\nActual labels:\n\\n\n{\ny\n.\nflatten\n()\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding the Sigmoid Function\ndef\nsigmoid\n(\nx\n):\n\"\"\"\nSigmoid activation function\n\"\"\"\nreturn\n1\n/\n(\n1\n+\nnp\n.\nexp\n(\n-\nnp\n.\nclip\n(\nx\n,\n-\n500\n,\n500\n)))\n# Clip to prevent overflow\n# Test sigmoid on different inputs\ntest_values\n=\nnp\n.\narray\n([\n-\n10\n,\n-\n1\n,\n0\n,\n1\n,\n10\n])\nsigmoid_values\n=\nsigmoid\n(\ntest_values\n)\nprint\n(\n\"\nSigmoid function behavior:\n\"\n)\nfor\ni\n,\nval\nin\nenumerate\n(\ntest_values\n):\nprint\n(\nf\n\"\nsigmoid(\n{\nval\n:\n3.0\nf\n}\n) =\n{\nsigmoid_values\n[\ni\n]\n:\n.\n3\nf\n}\n\"\n)\n# Visualize sigmoid\nx_range\n=\nnp\n.\nlinspace\n(\n-\n10\n,\n10\n,\n100\n)\ny_sigmoid\n=\nsigmoid\n(\nx_range\n)\nplt\n.\nfigure\n(\nfigsize\n=\n(\n8\n,\n4\n))\nplt\n.\nplot\n(\nx_range\n,\ny_sigmoid\n,\n'\nb-\n'\n,\nlinewidth\n=\n2\n)\nplt\n.\ntitle\n(\n'\nSigmoid Activation Function\n'\n)\nplt\n.\nxlabel\n(\n'\nx\n'\n)\nplt\n.\nylabel\n(\n'\nsigmoid(x)\n'\n)\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\nshow\n()\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Breaking Down the Sigmoid Formula\n# Let's understand sigmoid step by step: 1 / (1 + e^(-x))\nx\n=\n2.0\nprint\n(\nf\n\"\nInput: x =\n{\nx\n}\n\"\n)\nprint\n(\nf\n\"\nStep 1: -x =\n{\n-\nx\n}\n\"\n)\nprint\n(\nf\n\"\nStep 2: e^(-x) = np.exp(-x) =\n{\nnp\n.\nexp\n(\n-\nx\n)\n:\n.\n3\nf\n}\n\"\n)\nprint\n(\nf\n\"\nStep 3: 1 + e^(-x) =\n{\n1\n+\nnp\n.\nexp\n(\n-\nx\n)\n:\n.\n3\nf\n}\n\"\n)\nprint\n(\nf\n\"\nStep 4: 1 / (1 + e^(-x)) =\n{\n1\n/\n(\n1\n+\nnp\n.\nexp\n(\n-\nx\n))\n:\n.\n3\nf\n}\n\"\n)\nprint\n(\nf\n\"\nSigmoid result:\n{\nsigmoid\n(\nx\n)\n:\n.\n3\nf\n}\n\"\n)\n# Why clipping?\nprint\n(\nf\n\"\n\\n\nWhy we clip large values:\n\"\n)\nlarge_x\n=\n1000\nprint\n(\nf\n\"\nWithout clipping: np.exp(-\n{\nlarge_x\n}\n) would cause overflow\n\"\n)\nprint\n(\nf\n\"\nWith clipping: np.exp(-500) =\n{\nnp\n.\nexp\n(\n-\n500\n)\n:\n.\n2\ne\n}\n(very small, safe)\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nKey insight\n: The forward pass transforms input through weighted connections and activations to produce predictions.\nPart 3: Computing Loss and Gradients\nLoss Function\ndef\ncompute_loss\n(\npredictions\n,\ntargets\n):\n\"\"\"\nMean squared error loss\n\"\"\"\nreturn\nnp\n.\nmean\n((\npredictions\n-\ntargets\n)\n**\n2\n)\n# Square the difference to penalize large errors\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Why We Square the Difference\n# Let's see why we use (predictions - targets) ** 2\npred\n=\nnp\n.\narray\n([\n0.8\n,\n0.2\n,\n0.9\n,\n0.1\n])\ntarget\n=\nnp\n.\narray\n([\n1.0\n,\n0.0\n,\n1.0\n,\n0.0\n])\ndifferences\n=\npred\n-\ntarget\nsquared_differences\n=\ndifferences\n**\n2\nprint\n(\n\"\nUnderstanding squared error:\n\"\n)\nprint\n(\nf\n\"\nPredictions:\n{\npred\n}\n\"\n)\nprint\n(\nf\n\"\nTargets:\n{\ntarget\n}\n\"\n)\nprint\n(\nf\n\"\nDifferences:\n{\ndifferences\n}\n\"\n)\nprint\n(\nf\n\"\nSquared:\n{\nsquared_differences\n}\n\"\n)\nprint\n(\nf\n\"\nMean squared error:\n{\nnp\n.\nmean\n(\nsquared_differences\n)\n:\n.\n4\nf\n}\n\"\n)\n# Why not just absolute difference?\nabs_differences\n=\nnp\n.\nabs\n(\ndifferences\n)\nprint\n(\nf\n\"\n\\n\nComparison:\n\"\n)\nprint\n(\nf\n\"\nAbsolute differences:\n{\nabs_differences\n}\n\"\n)\nprint\n(\nf\n\"\nSquared differences:\n{\nsquared_differences\n}\n\"\n)\nprint\n(\n\"\nSquared errors penalize large mistakes more heavily!\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nCalculate initial loss\ninitial_loss\n=\ncompute_loss\n(\npredictions\n,\ny\n)\nprint\n(\nf\n\"\nInitial loss:\n{\ninitial_loss\n:\n.\n4\nf\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Derivative of Sigmoid\ndef\nsigmoid_derivative\n(\nx\n):\n\"\"\"\nDerivative of sigmoid function\n\"\"\"\ns\n=\nsigmoid\n(\nx\n)\nreturn\ns\n*\n(\n1\n-\ns\n)\n# Derivative formula: sigmoid(x) * (1 - sigmoid(x))\n# Test derivative\ntest_vals\n=\nnp\n.\narray\n([\n-\n2\n,\n-\n1\n,\n0\n,\n1\n,\n2\n])\nsigmoid_vals\n=\nsigmoid\n(\ntest_vals\n)\nderivative_vals\n=\nsigmoid_derivative\n(\ntest_vals\n)\nprint\n(\n\"\nSigmoid and its derivative:\n\"\n)\nfor\ni\n,\nval\nin\nenumerate\n(\ntest_vals\n):\nprint\n(\nf\n\"\nx=\n{\nval\n:\n2.0\nf\n}\n: sigmoid=\n{\nsigmoid_vals\n[\ni\n]\n:\n.\n3\nf\n}\n, derivative=\n{\nderivative_vals\n[\ni\n]\n:\n.\n3\nf\n}\n\"\n)\n# Visualize both functions\nx_range\n=\nnp\n.\nlinspace\n(\n-\n6\n,\n6\n,\n100\n)\ny_sigmoid\n=\nsigmoid\n(\nx_range\n)\ny_derivative\n=\nsigmoid_derivative\n(\nx_range\n)\nplt\n.\nfigure\n(\nfigsize\n=\n(\n10\n,\n4\n))\nplt\n.\nplot\n(\nx_range\n,\ny_sigmoid\n,\n'\nb-\n'\n,\nlabel\n=\n'\nsigmoid(x)\n'\n,\nlinewidth\n=\n2\n)\nplt\n.\nplot\n(\nx_range\n,\ny_derivative\n,\n'\nr--\n'\n,\nlabel\n=\n\"\nsigmoid\n'\n(x)\n\"\n,\nlinewidth\n=\n2\n)\nplt\n.\ntitle\n(\n'\nSigmoid Function and Its Derivative\n'\n)\nplt\n.\nxlabel\n(\n'\nx\n'\n)\nplt\n.\nylabel\n(\n'\ny\n'\n)\nplt\n.\nlegend\n()\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\nshow\n()\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Chain Rule in Backpropagation\n# The chain rule: if y = f(g(x)), then dy/dx = f'(g(x)) * g'(x)\n# \n# For our network: Loss = MSE(sigmoid(W2 * sigmoid(W1 * X + b1) + b2), y)\n# We need: dLoss/dW2, dLoss/db2, dLoss/dW1, dLoss/db1\nprint\n(\n\"\nChain rule breakdown:\n\"\n)\nprint\n(\n\"\ndLoss/dW2 = dLoss/da2 * da2/dz2 * dz2/dW2\n\"\n)\nprint\n(\n\"\nwhere:\n\"\n)\nprint\n(\n\"\ndLoss/da2 = 2 * (predictions - targets)  # MSE derivative\n\"\n)\nprint\n(\n\"\nda2/dz2 = sigmoid\n'\n(z2)                   # sigmoid derivative\n\"\n)\nprint\n(\n\"\ndz2/dW2 = a1                             # linear layer derivative\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nBackpropagation Implementation\ndef\nbackward_pass\n(\nX\n,\ny\n,\nz1\n,\na1\n,\nz2\n,\na2\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n):\n\"\"\"\nCompute gradients using backpropagation\n\"\"\"\nm\n=\nX\n.\nshape\n[\n0\n]\n# Number of samples\n# Output layer gradients\ndz2\n=\n2\n*\n(\na2\n-\ny\n)\n*\nsigmoid_derivative\n(\nz2\n)\n# (4, 1)\ndW2\n=\na1\n.\nT\n@\ndz2\n/\nm\n# (4, 1)\ndb2\n=\nnp\n.\nmean\n(\ndz2\n,\naxis\n=\n0\n,\nkeepdims\n=\nTrue\n)\n# (1, 1)\n# Hidden layer gradients\ndz1\n=\n(\ndz2\n@\nW2\n.\nT\n)\n*\nsigmoid_derivative\n(\nz1\n)\n# (4, 4)\ndW1\n=\nX\n.\nT\n@\ndz1\n/\nm\n# (2, 4)\ndb1\n=\nnp\n.\nmean\n(\ndz1\n,\naxis\n=\n0\n,\nkeepdims\n=\nTrue\n)\n# (1, 4)\nreturn\ndW1\n,\ndb1\n,\ndW2\n,\ndb2\n# Test backpropagation\ndW1\n,\ndb1\n,\ndW2\n,\ndb2\n=\nbackward_pass\n(\nX\n,\ny\n,\nz1\n,\na1\n,\nz2\n,\npredictions\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n)\nprint\n(\nf\n\"\nGradient shapes:\n\"\n)\nprint\n(\nf\n\"\ndW1:\n{\ndW1\n.\nshape\n}\n, dW2:\n{\ndW2\n.\nshape\n}\n\"\n)\nprint\n(\nf\n\"\ndb1:\n{\ndb1\n.\nshape\n}\n, db2:\n{\ndb2\n.\nshape\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Output Layer Gradients\n# Let's break down the output layer gradient calculation\nprint\n(\n\"\nOutput layer gradient breakdown:\n\"\n)\nprint\n(\n\"\ndz2 = 2 * (a2 - y) * sigmoid_derivative(z2)\n\"\n)\n# Step by step\nerror\n=\na2\n-\ny\n# How far off our predictions are\nprint\n(\nf\n\"\nError (a2 - y) shape:\n{\nerror\n.\nshape\n}\n\"\n)\nprint\n(\nf\n\"\nError values:\n\\n\n{\nerror\n.\nflatten\n()\n}\n\"\n)\nmse_gradient\n=\n2\n*\nerror\n# Derivative of MSE\nprint\n(\nf\n\"\n\\n\nMSE gradient (2 * error) shape:\n{\nmse_gradient\n.\nshape\n}\n\"\n)\nsigmoid_grad\n=\nsigmoid_derivative\n(\nz2\n)\n# Derivative of sigmoid\nprint\n(\nf\n\"\nSigmoid gradient shape:\n{\nsigmoid_grad\n.\nshape\n}\n\"\n)\ndz2_step\n=\nmse_gradient\n*\nsigmoid_grad\n# Chain rule\nprint\n(\nf\n\"\nCombined gradient (dz2) shape:\n{\ndz2_step\n.\nshape\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Hidden Layer Gradients\n# Hidden layer gradients are more complex due to chain rule\nprint\n(\n\"\nHidden layer gradient breakdown:\n\"\n)\nprint\n(\n\"\ndz1 = (dz2 @ W2.T) * sigmoid_derivative(z1)\n\"\n)\n# Step by step\nerror_propagated\n=\ndz2\n@\nW2\n.\nT\n# Propagate error backwards\nprint\n(\nf\n\"\nError propagated shape:\n{\nerror_propagated\n.\nshape\n}\n\"\n)\nprint\n(\nf\n\"\nThis spreads output error to each hidden neuron\n\"\n)\nhidden_sigmoid_grad\n=\nsigmoid_derivative\n(\nz1\n)\n# Local gradient\nprint\n(\nf\n\"\nHidden sigmoid gradient shape:\n{\nhidden_sigmoid_grad\n.\nshape\n}\n\"\n)\ndz1_step\n=\nerror_propagated\n*\nhidden_sigmoid_grad\n# Final gradient\nprint\n(\nf\n\"\nCombined hidden gradient (dz1) shape:\n{\ndz1_step\n.\nshape\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Weight Gradients\n# Weight gradients show how to adjust connections\nprint\n(\n\"\nWeight gradient calculation:\n\"\n)\nprint\n(\n\"\ndW2 = a1.T @ dz2 / m\n\"\n)\nprint\n(\nf\n\"\na1.T shape:\n{\na1\n.\nT\n.\nshape\n}\n\"\n)\n# Transposed hidden activations\nprint\n(\nf\n\"\ndz2 shape:\n{\ndz2\n.\nshape\n}\n\"\n)\n# Output gradients\nprint\n(\nf\n\"\ndW2 shape:\n{\n(\na1\n.\nT\n@\ndz2\n).\nshape\n}\n\"\n)\n# Weight gradients\n# This gives us the gradient for each weight connection\nprint\n(\nf\n\"\n\\n\nWeight gradients tell us:\n\"\n)\nprint\n(\nf\n\"\n- Positive gradient: decrease this weight\n\"\n)\nprint\n(\nf\n\"\n- Negative gradient: increase this weight\n\"\n)\nprint\n(\nf\n\"\n- Large gradient: this weight has big impact on error\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nCritical concept\n: Backpropagation uses the chain rule to compute how much each weight contributes to the total error.\nPart 4: Training the Network\nTraining Loop\ndef\ntrain_network\n(\nX\n,\ny\n,\nepochs\n=\n1000\n,\nlearning_rate\n=\n1.0\n):\n\"\"\"\nTrain the neural network\n\"\"\"\n# Initialize weights\nnp\n.\nrandom\n.\nseed\n(\n42\n)\nW1\n=\nnp\n.\nrandom\n.\nrandn\n(\n2\n,\n4\n)\n*\n0.5\nb1\n=\nnp\n.\nzeros\n((\n1\n,\n4\n))\nW2\n=\nnp\n.\nrandom\n.\nrandn\n(\n4\n,\n1\n)\n*\n0.5\nb2\n=\nnp\n.\nzeros\n((\n1\n,\n1\n))\nlosses\n=\n[]\nfor\nepoch\nin\nrange\n(\nepochs\n):\n# Forward pass\nz1\n,\na1\n,\nz2\n,\npredictions\n=\nforward_pass\n(\nX\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n)\n# Compute loss\nloss\n=\ncompute_loss\n(\npredictions\n,\ny\n)\nlosses\n.\nappend\n(\nloss\n)\n# Backward pass\ndW1\n,\ndb1\n,\ndW2\n,\ndb2\n=\nbackward_pass\n(\nX\n,\ny\n,\nz1\n,\na1\n,\nz2\n,\npredictions\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n)\n# Update weights\nW1\n-=\nlearning_rate\n*\ndW1\nb1\n-=\nlearning_rate\n*\ndb1\nW2\n-=\nlearning_rate\n*\ndW2\nb2\n-=\nlearning_rate\n*\ndb2\n# Print progress\nif\nepoch\n%\n100\n==\n0\n:\nprint\n(\nf\n\"\nEpoch\n{\nepoch\n:\n4\nd\n}\n: Loss =\n{\nloss\n:\n.\n6\nf\n}\n\"\n)\nreturn\nW1\n,\nb1\n,\nW2\n,\nb2\n,\nlosses\n# Train the network\nW1_trained\n,\nb1_trained\n,\nW2_trained\n,\nb2_trained\n,\nloss_history\n=\ntrain_network\n(\nX\n,\ny\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Understanding Learning Rate\n# Learning rate controls how big steps we take during optimization\n# Too small: slow convergence, too large: might overshoot minimum\nlearning_rates\n=\n[\n0.1\n,\n1.0\n,\n10.0\n]\nplt\n.\nfigure\n(\nfigsize\n=\n(\n12\n,\n4\n))\nfor\ni\n,\nlr\nin\nenumerate\n(\nlearning_rates\n):\nplt\n.\nsubplot\n(\n1\n,\n3\n,\ni\n+\n1\n)\n# Train with this learning rate\n_\n,\n_\n,\n_\n,\n_\n,\nlosses\n=\ntrain_network\n(\nX\n,\ny\n,\nepochs\n=\n500\n,\nlearning_rate\n=\nlr\n)\nplt\n.\nplot\n(\nlosses\n)\nplt\n.\ntitle\n(\nf\n'\nLearning Rate =\n{\nlr\n}\n'\n)\nplt\n.\nxlabel\n(\n'\nEpoch\n'\n)\nplt\n.\nylabel\n(\n'\nLoss\n'\n)\nplt\n.\nyscale\n(\n'\nlog\n'\n)\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\ntight_layout\n()\nplt\n.\nshow\n()\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Visualizing Training Progress\n# Plot loss curve\nplt\n.\nfigure\n(\nfigsize\n=\n(\n10\n,\n4\n))\nplt\n.\nsubplot\n(\n1\n,\n2\n,\n1\n)\nplt\n.\nplot\n(\nloss_history\n)\nplt\n.\ntitle\n(\n'\nTraining Loss\n'\n)\nplt\n.\nxlabel\n(\n'\nEpoch\n'\n)\nplt\n.\nylabel\n(\n'\nLoss\n'\n)\nplt\n.\nyscale\n(\n'\nlog\n'\n)\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\nsubplot\n(\n1\n,\n2\n,\n2\n)\nplt\n.\nplot\n(\nloss_history\n[\n-\n100\n:])\n# Last 100 epochs\nplt\n.\ntitle\n(\n'\nTraining Loss (Last 100 Epochs)\n'\n)\nplt\n.\nxlabel\n(\n'\nEpoch\n'\n)\nplt\n.\nylabel\n(\n'\nLoss\n'\n)\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\ntight_layout\n()\nplt\n.\nshow\n()\nprint\n(\nf\n\"\nFinal loss:\n{\nloss_history\n[\n-\n1\n]\n:\n.\n6\nf\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nPart 5: Testing the Trained Network\nFinal Predictions\n# Test the trained network\nz1_final\n,\na1_final\n,\nz2_final\n,\nfinal_predictions\n=\nforward_pass\n(\nX\n,\nW1_trained\n,\nb1_trained\n,\nW2_trained\n,\nb2_trained\n)\nprint\n(\n\"\nFinal Results:\n\"\n)\nprint\n(\n\"\nInput -> Target | Prediction | Rounded\n\"\n)\nprint\n(\n\"\n-\n\"\n*\n40\n)\nfor\ni\nin\nrange\n(\nlen\n(\nX\n)):\npred\n=\nfinal_predictions\n[\ni\n,\n0\n]\nrounded\n=\nround\n(\npred\n)\ntarget\n=\ny\n[\ni\n,\n0\n]\nprint\n(\nf\n\"\n{\nX\n[\ni\n]\n}\n->\n{\ntarget\n}\n|\n{\npred\n:\n.\n4\nf\n}\n|\n{\nrounded\n}\n\"\n)\n# Calculate accuracy\nrounded_predictions\n=\nnp\n.\nround\n(\nfinal_predictions\n)\naccuracy\n=\nnp\n.\nmean\n(\nrounded_predictions\n==\ny\n)\nprint\n(\nf\n\"\n\\n\nAccuracy:\n{\naccuracy\n:\n.\n1\n%\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Visualizing Decision Boundary\n# Create a grid of points to visualize the decision boundary\ndef\nplot_decision_boundary\n(\nW1\n,\nb1\n,\nW2\n,\nb2\n):\n\"\"\"\nPlot the decision boundary learned by the network\n\"\"\"\n# Create a grid\nxx\n,\nyy\n=\nnp\n.\nmeshgrid\n(\nnp\n.\nlinspace\n(\n-\n0.5\n,\n1.5\n,\n100\n),\nnp\n.\nlinspace\n(\n-\n0.5\n,\n1.5\n,\n100\n))\n# Flatten the grid for prediction\ngrid_points\n=\nnp\n.\nc_\n[\nxx\n.\nravel\n(),\nyy\n.\nravel\n()]\n# Make predictions on the grid\n_\n,\n_\n,\n_\n,\ngrid_predictions\n=\nforward_pass\n(\ngrid_points\n,\nW1\n,\nb1\n,\nW2\n,\nb2\n)\ngrid_predictions\n=\ngrid_predictions\n.\nreshape\n(\nxx\n.\nshape\n)\n# Plot\nplt\n.\nfigure\n(\nfigsize\n=\n(\n8\n,\n6\n))\nplt\n.\ncontourf\n(\nxx\n,\nyy\n,\ngrid_predictions\n,\nlevels\n=\n50\n,\nalpha\n=\n0.8\n,\ncmap\n=\n'\nRdYlBu\n'\n)\nplt\n.\ncolorbar\n(\nlabel\n=\n'\nNetwork Output\n'\n)\n# Plot data points\ncolors\n=\n[\n'\nred\n'\nif\nlabel\n==\n0\nelse\n'\nblue\n'\nfor\nlabel\nin\ny\n.\nflatten\n()]\nplt\n.\nscatter\n(\nX\n[:,\n0\n],\nX\n[:,\n1\n],\nc\n=\ncolors\n,\ns\n=\n100\n,\nedgecolors\n=\n'\nblack\n'\n,\nlinewidth\n=\n2\n)\n# Add labels\nfor\ni\n,\n(\nx\n,\ny_val\n)\nin\nenumerate\n(\nzip\n(\nX\n,\ny\n.\nflatten\n())):\nplt\n.\nannotate\n(\nf\n'\n(\n{\nx\n[\n0\n]\n}\n,\n{\nx\n[\n1\n]\n}\n)→\n{\ny_val\n}\n'\n,\n(\nx\n[\n0\n],\nx\n[\n1\n]),\nxytext\n=\n(\n5\n,\n5\n),\ntextcoords\n=\n'\noffset points\n'\n)\nplt\n.\ntitle\n(\n'\nNeural Network Decision Boundary\n'\n)\nplt\n.\nxlabel\n(\n'\nInput 1\n'\n)\nplt\n.\nylabel\n(\n'\nInput 2\n'\n)\nplt\n.\ngrid\n(\nTrue\n,\nalpha\n=\n0.3\n)\nplt\n.\nshow\n()\n# Visualize the decision boundary\nplot_decision_boundary\n(\nW1_trained\n,\nb1_trained\n,\nW2_trained\n,\nb2_trained\n)\nEnter fullscreen mode\nExit fullscreen mode\nPart 6: Understanding What We Built\nComplete Neural Network Class\nclass\nSimpleNeuralNetwork\n:\n\"\"\"\nA simple 2-layer neural network implementation\n\"\"\"\ndef\n__init__\n(\nself\n,\ninput_size\n=\n2\n,\nhidden_size\n=\n4\n,\noutput_size\n=\n1\n):\n# Initialize weights\nself\n.\nW1\n=\nnp\n.\nrandom\n.\nrandn\n(\ninput_size\n,\nhidden_size\n)\n*\n0.5\nself\n.\nb1\n=\nnp\n.\nzeros\n((\n1\n,\nhidden_size\n))\nself\n.\nW2\n=\nnp\n.\nrandom\n.\nrandn\n(\nhidden_size\n,\noutput_size\n)\n*\n0.5\nself\n.\nb2\n=\nnp\n.\nzeros\n((\n1\n,\noutput_size\n))\ndef\nsigmoid\n(\nself\n,\nx\n):\nreturn\n1\n/\n(\n1\n+\nnp\n.\nexp\n(\n-\nnp\n.\nclip\n(\nx\n,\n-\n500\n,\n500\n)))\ndef\nforward\n(\nself\n,\nX\n):\nself\n.\nz1\n=\nX\n@\nself\n.\nW1\n+\nself\n.\nb1\nself\n.\na1\n=\nself\n.\nsigmoid\n(\nself\n.\nz1\n)\nself\n.\nz2\n=\nself\n.\na1\n@\nself\n.\nW2\n+\nself\n.\nb2\nself\n.\na2\n=\nself\n.\nsigmoid\n(\nself\n.\nz2\n)\nreturn\nself\n.\na2\ndef\nbackward\n(\nself\n,\nX\n,\ny\n):\nm\n=\nX\n.\nshape\n[\n0\n]\n# Output layer gradients\ndz2\n=\n2\n*\n(\nself\n.\na2\n-\ny\n)\n*\nself\n.\nsigmoid\n(\nself\n.\nz2\n)\n*\n(\n1\n-\nself\n.\nsigmoid\n(\nself\n.\nz2\n))\ndW2\n=\nself\n.\na1\n.\nT\n@\ndz2\n/\nm\ndb2\n=\nnp\n.\nmean\n(\ndz2\n,\naxis\n=\n0\n,\nkeepdims\n=\nTrue\n)\n# Hidden layer gradients\ndz1\n=\n(\ndz2\n@\nself\n.\nW2\n.\nT\n)\n*\nself\n.\nsigmoid\n(\nself\n.\nz1\n)\n*\n(\n1\n-\nself\n.\nsigmoid\n(\nself\n.\nz1\n))\ndW1\n=\nX\n.\nT\n@\ndz1\n/\nm\ndb1\n=\nnp\n.\nmean\n(\ndz1\n,\naxis\n=\n0\n,\nkeepdims\n=\nTrue\n)\nreturn\ndW1\n,\ndb1\n,\ndW2\n,\ndb2\ndef\ntrain\n(\nself\n,\nX\n,\ny\n,\nepochs\n=\n1000\n,\nlearning_rate\n=\n1.0\n):\nlosses\n=\n[]\nfor\nepoch\nin\nrange\n(\nepochs\n):\n# Forward pass\npredictions\n=\nself\n.\nforward\n(\nX\n)\n# Compute loss\nloss\n=\nnp\n.\nmean\n((\npredictions\n-\ny\n)\n**\n2\n)\nlosses\n.\nappend\n(\nloss\n)\n# Backward pass\ndW1\n,\ndb1\n,\ndW2\n,\ndb2\n=\nself\n.\nbackward\n(\nX\n,\ny\n)\n# Update weights\nself\n.\nW1\n-=\nlearning_rate\n*\ndW1\nself\n.\nb1\n-=\nlearning_rate\n*\ndb1\nself\n.\nW2\n-=\nlearning_rate\n*\ndW2\nself\n.\nb2\n-=\nlearning_rate\n*\ndb2\nif\nepoch\n%\n100\n==\n0\n:\nprint\n(\nf\n\"\nEpoch\n{\nepoch\n:\n4\nd\n}\n: Loss =\n{\nloss\n:\n.\n6\nf\n}\n\"\n)\nreturn\nlosses\ndef\npredict\n(\nself\n,\nX\n):\nreturn\nself\n.\nforward\n(\nX\n)\n# Test the class\nnn\n=\nSimpleNeuralNetwork\n()\nlosses\n=\nnn\n.\ntrain\n(\nX\n,\ny\n,\nepochs\n=\n1000\n,\nlearning_rate\n=\n1.0\n)\npredictions\n=\nnn\n.\npredict\n(\nX\n)\nprint\n(\n\"\n\\n\nClass-based Neural Network Results:\n\"\n)\nfor\ni\nin\nrange\n(\nlen\n(\nX\n)):\npred\n=\npredictions\n[\ni\n,\n0\n]\ntarget\n=\ny\n[\ni\n,\n0\n]\nprint\n(\nf\n\"\n{\nX\n[\ni\n]\n}\n->\n{\ntarget\n}\n| Prediction:\n{\npred\n:\n.\n4\nf\n}\n| Rounded:\n{\nround\n(\npred\n)\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nOPTIONAL: Comparing with Different Architectures\n# Test different hidden layer sizes\nhidden_sizes\n=\n[\n2\n,\n4\n,\n8\n,\n16\n]\nresults\n=\n{}\nfor\nhidden_size\nin\nhidden_sizes\n:\nprint\n(\nf\n\"\n\\n\nTesting hidden size:\n{\nhidden_size\n}\n\"\n)\nnn\n=\nSimpleNeuralNetwork\n(\ninput_size\n=\n2\n,\nhidden_size\n=\nhidden_size\n,\noutput_size\n=\n1\n)\nlosses\n=\nnn\n.\ntrain\n(\nX\n,\ny\n,\nepochs\n=\n1000\n,\nlearning_rate\n=\n1.0\n)\npredictions\n=\nnn\n.\npredict\n(\nX\n)\n# Calculate accuracy\naccuracy\n=\nnp\n.\nmean\n(\nnp\n.\nround\n(\npredictions\n)\n==\ny\n)\nresults\n[\nhidden_size\n]\n=\n{\n'\nfinal_loss\n'\n:\nlosses\n[\n-\n1\n],\n'\naccuracy\n'\n:\naccuracy\n,\n'\npredictions\n'\n:\npredictions\n}\nprint\n(\nf\n\"\nFinal loss:\n{\nlosses\n[\n-\n1\n]\n:\n.\n6\nf\n}\n\"\n)\nprint\n(\nf\n\"\nAccuracy:\n{\naccuracy\n:\n.\n1\n%\n}\n\"\n)\n# Plot comparison\nplt\n.\nfigure\n(\nfigsize\n=\n(\n12\n,\n4\n))\nplt\n.\nsubplot\n(\n1\n,\n2\n,\n1\n)\nhidden_sizes_list\n=\nlist\n(\nresults\n.\nkeys\n())\nfinal_losses\n=\n[\nresults\n[\nhs\n][\n'\nfinal_loss\n'\n]\nfor\nhs\nin\nhidden_sizes_list\n]\nplt\n.\nbar\n(\nhidden_sizes_list\n,\nfinal_losses\n)\nplt\n.\ntitle\n(\n'\nFinal Loss vs Hidden Size\n'\n)\nplt\n.\nxlabel\n(\n'\nHidden Layer Size\n'\n)\nplt\n.\nylabel\n(\n'\nFinal Loss\n'\n)\nplt\n.\nyscale\n(\n'\nlog\n'\n)\nplt\n.\nsubplot\n(\n1\n,\n2\n,\n2\n)\naccuracies\n=\n[\nresults\n[\nhs\n][\n'\naccuracy\n'\n]\nfor\nhs\nin\nhidden_sizes_list\n]\nplt\n.\nbar\n(\nhidden_sizes_list\n,\naccuracies\n)\nplt\n.\ntitle\n(\n'\nAccuracy vs Hidden Size\n'\n)\nplt\n.\nxlabel\n(\n'\nHidden Layer Size\n'\n)\nplt\n.\nylabel\n(\n'\nAccuracy\n'\n)\nplt\n.\nylim\n(\n0\n,\n1\n)\nplt\n.\ntight_layout\n()\nplt\n.\nshow\n()\nEnter fullscreen mode\nExit fullscreen mode\nKey takeaway\n: You've built a complete neural network from scratch! The network learns to solve the XOR problem by discovering the right weights and biases through gradient descent.\nSummary\nYou've successfully implemented:\nForward propagation\n: Computing predictions from inputs\nLoss computation\n: Measuring how wrong the predictions are\nBackpropagation\n: Computing gradients using the chain rule\nWeight updates\n: Using gradient descent to improve the network\nComplete training loop\n: Putting it all together\nThe neural network learns by repeatedly adjusting its weights based on the errors it makes, eventually discovering the complex decision boundary needed to solve the XOR problem.",
  "metadata": {
    "title": "Code a Neural Network from Scratch in NumPy - DEV Community",
    "description": "Part of my Zero to AI Researcher / Engineer Course           Part 1: Getting Started - Your First...",
    "keywords": "software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/vuk_rosic/code-a-neural-network-from-scratch-in-numpy-494o",
    "og:title": "Code a Neural Network from Scratch in NumPy",
    "og:description": "Part of my Zero to AI Researcher / Engineer Course           Part 1: Getting Started - Your First...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmvtldzuomx8t78botksy.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@VukRosic99",
    "twitter:title": "Code a Neural Network from Scratch in NumPy",
    "twitter:description": "Part of my Zero to AI Researcher / Engineer Course           Part 1: Getting Started - Your First...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmvtldzuomx8t78botksy.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/e89b1bae3e6e/code/body-000.txt",
      "chars": 391,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-001.txt",
      "chars": 816,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-002.txt",
      "chars": 558,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-003.txt",
      "chars": 579,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-004.txt",
      "chars": 669,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-005.txt",
      "chars": 551,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-006.txt",
      "chars": 173,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-007.txt",
      "chars": 711,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-008.txt",
      "chars": 87,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-009.txt",
      "chars": 910,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-010.txt",
      "chars": 522,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-011.txt",
      "chars": 813,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-012.txt",
      "chars": 650,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-013.txt",
      "chars": 614,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-014.txt",
      "chars": 567,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-015.txt",
      "chars": 1030,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-016.txt",
      "chars": 567,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-017.txt",
      "chars": 459,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-018.txt",
      "chars": 556,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-019.txt",
      "chars": 1348,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-020.txt",
      "chars": 2285,
      "language": "text"
    },
    {
      "file": "attachments/e89b1bae3e6e/code/body-021.txt",
      "chars": 1238,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://dev.to/vuk_rosic/zero-to-mastery-ai-researcher-engineer-in-development-amg",
      "label": "Zero to AI Researcher / Engineer Course",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:24Z",
  "source_url": "https://dev.to/vuk_rosic/code-a-neural-network-from-scratch-in-numpy-494o",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 20132,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/e89b1bae3e6e/pages/source.html",
  "local_html": "attachments/e89b1bae3e6e/pages/article.html",
  "body_base_path": "attachments/e89b1bae3e6e/pages/",
  "body_markdown_path": "attachments/e89b1bae3e6e/pages/article.md"
}