{ "source_file": "0930", "source_platform": "个人博客", "author": "Jay Alammar", "title_listed": "A Visual And Interactive Look at Basic Neural Network Math", "keywords": "神经网络,数学,可视化", "domain": "机器学习", "content_type": "图解教程", "is_paid_flag": "否", "url": "https://jalammar.github.io/feedforward-neural-networks-visual-interactive/", "publish_date": "2018-11", "corpus_stage": "预训练/SFT", "notes": "神经网络基础数学可视化", "id": "daeb388c0838", "collected_at": "2026-10-02T14:22:58", "title": "A Visual And Interactive Look at Basic Neural Network Math – Jay Alammar – Visualizing machine learning one concept at a time.", "body_markdown": "A Visual And Interactive Look at Basic Neural Network Math\nIn the\nprevious post, we looked at the basic concepts of neural networks\n. Let us now take another example as an excuse to guide us to explore some of the basic mathematical ideas involved in prediction with neural networks.\nYour browser does not support the video tag.\nIf you had been aboard the Titanic, would you have survived the sinking event? Let’s build a model to predict one’s odds of survival.\nThis will be a neural network model building on what we discussed in the\nprevious post\n, but will have a higher prediction accuracy because it utilizes hidden layers and activation functions.\nThe dataset we’ll use this time will be the\nTitanic passenger list\nfrom Kaggle. It lists the names and other information of the passengers and shows whether each passenger survived the sinking event or not.\nThe raw dataset looks like this:\nPassengerId\nSurvived\nPclass\nName\nSex\nAge\nSibSp\nParch\nTicket\nFare\nCabin\nEmbarked\n1\n0\n3\nBraund, Mr. Owen Harris\nmale\n22.0\n1\n0\nA/5 21171\n7.2500\nNaN\nS\n2\n1\n1\nCumings, Mrs. John Bradley (Florence Briggs Th…\nfemale\n38.0\n1\n0\nPC 17599\n71.2833\nC85\nC\n3\n1\n3\nHeikkinen, Miss. Laina\nfemale\n26.0\n0\n0\nSTON/O2. 3101282\n7.9250\nNaN\nS\nWe won’t bother with most of the columns for now. We’ll just use the sex and age columns as our features, and survival as our label that we’ll try to predict.\nAge\nSex\nSurvived?\n22\n0\n0\n38\n1\n1\n26\n1\n1\n… 891 rows total\nWe’ll attempt to build a network that predicts whether a passenger survived or not.\nNeural networks need their inputs to be numeric. So we had to change the sex column – male is now 0, female is 1. You’ll notice the dataset already uses something similar for the survival column – survived is 1, did not survive is 0.\nThe simplest neural network we can use to train to make this prediction looks like this:\nCalculating a prediction is done by plugging in a value for \"age\" and \"sex\". The calculation then flows from the left to the right.\n Before we can use this net for prediction, however, we'll have to run a \"training\" process that will give us the values for the weights (\nw\n) and bias (\nb\n).\nNote: we have slightly adjusted the way we represent the networks from the previous post. The bias node specifically is more commonly represented like this\nLet’s recap the elements that make up this network and how they work:\nAn input neuron is where we plug in an input value (e.g. the age of a person). It’s where the calculation starts. The outgoing connection and the rest of the graph tell us what other calculations we need to do to calculate a prediction.\nIf a connection has a weight, then the value is multiplied by that weight as it passes through it.\nconnection_output = weight * connection_input\nIf a neuron has inputs, it sums their value and sends that sum along its outgoing connection(s).\nnode_output = input_1 + input_2\nSigmoid\n#\nTo turn the network’s calculation into a probability value between 0 and 1, we have to pass the value from the output layer through a “sigmoid” formula. Sigmoid squashes the output value of a neuron to between 0 and 1 according to a specific curve.\nf(x)=\nWhere e is the mathematical constant approximately equal to 2.71828\ndef sigmoid(x):\n return 1/(1 + math.exp(-x))\n\noutput = sigmoid(value)\nSigmoid Visualization\n#\n0\nInteract a little with sigmoid to see how it transforms various values\n0\nf(\n0\n) =\n1\n1\n+\ne\n−\nx\n\\frac{1}{1 + e^{-x}}\n​\n1\n+\ne\n​\n−(\nx\n)\n​\n​\n​\n​\n1\n​\n​\n=\nTo bring it all together, calculating a prediction with this shallow network looks like this:\ndef sigmoid(x):\n return 1/(1 + math.exp(-x))\n\ndef calculate_prediction(age, sex, weight_1, weight_2, bias):\n\n # Multiply the inputs by their weights, sum the results up\n layer_2_node = age * weight_1 + sex * weight_2 + 1 * bias\n\n prediction = sigmoid(layer_2_node)\n return prediction\nNow that we know the structure of our network, we can train it using [gradient descent] running on the first 600 rows of the 891-row dataset. I will not be addressing the training process in this post because that’s a separate concern at the moment. For now, I just want you to be comfortable with how a trained network calculates a prediction. Once you get this intuition down, we’ll proceed to training in a future post.\nThe training process gives us the following values (with an accuracy of 73.20%):\nweight_1 = -0.016852 # Associated with \"Age\"\nweight_2 = 0.704039 # Associated with \"Sex\" (where male is 0, female is 1)\nbias = -0.116309\nIntuitively, the weights indicate how much their associated property contribute to the prediction – odds of survival improve the younger a person is (since a larger age multiplied by the negative weight value gives a bigger negative number). They improve more if the person is female.\nPrediction Calculation\n#\nThe trained network now looks like this:\n(hover or click on values in the table to see their individual predictions)\nAn accuracy of 73.20% isn’t very impressive. This is a case that can benefit from adding a hidden layer. Hidden layers give the model more capacity to represent more sophisticated prediction functions that may do a better job (\nDeep Learning ch.5 page 113\n).\nIt’s often useful to apply certain math functions to the weighted outputs. These are called “activation functions” because historically they translated the output of the neuron into either 1 (On/active) or 0 (Off).\ndef activation_function(x):\n # Do something to the value\n ...\n\nweighted_sum = weight * (input_1 + input_2)\noutput = activation_function(weighted_sum)\nActivation functions are vital for hidden layers. Without them, deep networks would be no better than a shallow linear network. Read the “Commonly used activation functions” section from\nNeural Networks Part 1: Setting up the Architecture\nfor a look at various activation functions.\nReLU\n#\nA leading choice for activation function is called ReLU. It returns 0 if its input is negative, returns the number itself otherwise. Very simple!\nf(x) = max(0, x)\n# Naive scalar relu implementation. In the real world, most calculations are done on vectors\ndef relu(x):\n if x < 0:\n return 0\n else:\n return x\n\n\noutput = relu(value)\nReLU Visualization\n#\n0\nInteract a little with relu to see how it transforms various values\n0\nf(\n0\n) =\nmax( 0,\nx\n)\n=\nClosing\nThis post has been parked for more than a year. I had attempted to visualize a deeper network after this point, but that never materialized. I hope you enjoyed it. Let me know on\n@JayAlammar on Twitter\nif you have any feedback.\nWritten on March 23, 2018", "metadata": { "title": "A Visual And Interactive Look at Basic Neural Network Math – Jay Alammar – Visualizing machine learning one concept at a time.", "description": "In the previous post, we looked at the basic concepts of neural networks. Let us now take another example as an excuse to guide us to explore some of the basic mathematical ideas involved in prediction with neural networks.\n\n\n \n Your browser does not support the video tag.", "keywords": null, "author": "Jay Alammar", "og:description": "In the previous post, we looked at the basic concepts of neural networks. Let us now take another example as an excuse to guide us to explore some of the basic mathematical ideas involved in prediction with neural networks.\n\n\n \n Your browser does not support the video tag.", "og:title": "A Visual And Interactive Look at Basic Neural Network Math" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/daeb388c0838/code/body-000.txt", "chars": 46, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-001.txt", "chars": 32, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-002.txt", "chars": 73, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-003.txt", "chars": 299, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-004.txt", "chars": 148, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-005.txt", "chars": 156, "language": "text" }, { "file": "attachments/daeb388c0838/code/body-006.txt", "chars": 187, "language": "text" } ], "images": [ { "status": "ok", "url": "https://jalammar.github.io/images/two-input-one-output-sigmoid-network.png", "bytes": 46625, "reused": false, "local_path": "attachments/daeb388c0838/images/body-e7f29465dfc172ae319d.png", "src": "https://jalammar.github.io/images/two-input-one-output-sigmoid-network.png", "alt": "neural netowrk with two inputs, one output, and sigmoid output activation", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/input-neuron.png", "bytes": 3842, "reused": false, "local_path": "attachments/daeb388c0838/images/body-9dc55d4bde305130326f.png", "src": "https://jalammar.github.io/images/input-neuron.png", "alt": "input neuron", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/weight.png", "bytes": 3919, "reused": false, "local_path": "attachments/daeb388c0838/images/body-ef71c32e66e98e371857.png", "src": "https://jalammar.github.io/images/weight.png", "alt": "weighted neuron image", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/neuron.png", "bytes": 5661, "reused": false, "local_path": "attachments/daeb388c0838/images/body-a91baef8e8e980479f1b.png", "src": "https://jalammar.github.io/images/neuron.png", "alt": " neuron image", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/sigmoid-neuron.png", "bytes": 2185, "reused": false, "local_path": "attachments/daeb388c0838/images/body-2ec15037d1810749c4bb.png", "src": "https://jalammar.github.io/images/sigmoid-neuron.png", "alt": "sigmoid neuron", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/neuron_with_activation.png", "bytes": 6263, "reused": false, "local_path": "attachments/daeb388c0838/images/body-f68db1775d452243da48.png", "src": "https://jalammar.github.io/images/neuron_with_activation.png", "alt": "weighted neuron with activation", "downloaded": true }, { "status": "ok", "url": "https://jalammar.github.io/images/relu.png", "bytes": 1923, "reused": false, "local_path": "attachments/daeb388c0838/images/body-4bc94335600d7a72c8b4.png", "src": "https://jalammar.github.io/images/relu.png", "alt": "weighted neuron with activation", "downloaded": true } ], "audio": [], "videos": [ { "url": "https://jalammar.github.io/images/titanic_nn_calculation.mp4", "src": "https://jalammar.github.io/images/titanic_nn_calculation.mp4", "status": "invalid_or_wrong_resolution", "kind": "direct_video", "bytes": 653602, "reused": true, "content_type": "video/mp4", "valid": true, "height": 664, "width": 1138, "duration": "15.199667", "local_path": "attachments/daeb388c0838/videos/repair-d61347b7025163286444.mp4", "title": "", "retried_at": "2026-10-20T17:55:56Z" } ], "links": [ { "href": "https://jalammar.github.io/visual-interactive-guide-basics-neural-networks/", "label": "previous post, we looked at the basic concepts of neural networks", "type": "a" }, { "href": "https://jalammar.github.io/images/titanic_nn_calculation.mp4", "label": "", "type": "source" }, { "href": "http://www.deeplearningbook.org/contents/ml.html", "label": "Deep Learning ch.5 page 113", "type": "a" }, { "href": "https://cs231n.github.io/neural-networks-1/", "label": "Neural Networks Part 1: Setting up the Architecture", "type": "a" }, { "href": "https://twitter.com/JayAlammar", "label": "@JayAlammar on Twitter", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:52:32Z", "source_url": "https://jalammar.github.io/feedforward-neural-networks-visual-interactive/", "papers": [], "actual_video_files": [], "body_chars": 6563, "expected_body_images": 7, "discovery_status": "ok", "delivery_status": "partial", "missing_reason": "invalid_or_wrong_resolution", "raw_html": "attachments/daeb388c0838/pages/source.html", "local_html": "attachments/daeb388c0838/pages/article.html", "body_base_path": "attachments/daeb388c0838/pages/", "body_markdown_path": "attachments/daeb388c0838/pages/article.md" }