{ "source_file": "0930", "source_platform": "Hashnode", "author": "Muhammad Fahad Bashir", "title_listed": "5. Understanding Loss Functions in Neural Networks", "keywords": "", "domain": "机器学习", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://mfahadbashir.hashnode.dev/5-understanding-loss-functions-in-neural-networks", "publish_date": "2025-04-01", "corpus_stage": "", "notes": "", "id": "3629d1909f0b", "collected_at": "2026-10-02T14:10:56", "title": "5. Understanding Loss Functions in Neural Networks", "body_markdown": "5. Understanding Loss Functions in Neural Networks\nUpdated\nApril 1, 2025\n•\n5\nmin read\n•\nView as Markdown\nM\nMuhammad Fahad Bashir\nMachine Learning, Artificial Intelligence\nPart of series\nDeep Learning Essentials\nWelcome! In this post, we will explore loss functions in neural networks. Previously, we covered an example of logistic regression to help understand the concepts.our series-\nDeep Learning Essential\n.\nWhat is a Loss Function?\nA\nloss function\nmeasures the difference between the actual value (y) and the predicted value (ŷ) of a neural network. The goal of training a neural network is to minimize this difference so that the model can make accurate predictions.\nExample\nIf\nŷ is close to y\n, the loss is\nclose to 0\n.\nIf\nŷ is far from y\n, the loss is\nlarge\n.\n🔹 Key Takeaway:\nThe loss function tells us how far our predictions are from the actual values.\nWhat is Training?\nTraining a neural network means teaching it to make correct predictions. Imagine you have many pictures of cats and dogs. You want the network to tell you if a new picture is a cat or a dog.\nTo do this, the network uses\nweights\nand\nbiases\n. These are numbers that help the network make its predictions. Think of weights as how important each pixel in the picture is. Bias is like a starting point.\nThe goal of training is to find the right weights and biases. When the network sees a cat picture, it should say \"cat\" with high confidence. If it sees a dog picture, it should say \"dog.\"\nParameters: Weights and Biases\nIn neural networks, weights and biases are called\nparameters\n. When we train the network, we adjust these parameters. We want the network's prediction (\ny-hat\n) to be as close as possible to the real answer (\ny\n).\ny-hat:\nThe network's guess.\ny:\nThe correct answer.\nIf y-hat is close to y, the network is doing a good job.\nHow Loss Functions Works\nThe network makes a prediction (y-hat).\nThe loss function compares y-hat to the real answer (y).\nThe loss function gives a number that shows how big the error is.\nThe network uses this number to change its weights and biases.\nThe network makes a new prediction, and the process repeats.\nWe want the loss function to be as small as possible. This means the network's predictions are close to the real answers.\nType of Loss Functions\nConvex Functions: A Special Type of loss function\nWhen choosing a loss function, we often use\nconvex functions\n. Convex functions have a special shape. They have only one lowest point, called the\nglobal optimum\n.\nWhy do we like convex functions? Because they make it easier to train the network. With a convex function, the network can find the best weights and biases without getting stuck.\nNegative Log Loss: A Common Choice\nA common loss function in logistic regression is\nnegative log loss\n. It is also called\nbinary cross-entropy loss\n.\nNegative log loss tells us the distance between y-hat and y. If y-hat is close to y, the loss is small. If y-hat is far from y, the loss is big.\nThe formula for negative log loss looks like this:\n- [y * log(y-hat) + (1 - y) * log(1 - y-hat)]\nThis formula might look scary, but it's not too bad. Let's break it down with examples.\nExample 1: y = 0, y-hat = 0\nIf the real answer is 0, and the network predicts 0, the loss is 0. This is good! The network is correct.\nExample 2: y = 1, y-hat = 0.1\nIf the real answer is 1, but the network predicts 0.1, the loss is higher. This tells us the network needs to improve.\nThe negative log loss penalizes incorrect predictions more heavily.\nCost Function: Average Loss\nWe use the loss function to measure the error for one sample. What if we have many samples? We need to find the average loss over all the samples.\nThis average loss is called the cost function.\nThe cost function tells us how well our network is doing on average. We want the cost function to be as small as possible.\nTo calculate the cost function:\nCalculate the loss for each sample.\nAdd up all the losses.\nDivide by the number of samples.\nThe cost function depends on the weights and biases. By changing the weights and biases, we can change the cost function. Our goal is to find the weights and biases that make the cost function as small as possible\nImproving Weights and Biases\nSo far, we've talked about how to measure the error of our network. But how do we actually improve the weights and biases?In\nlogistic regression\nand\nneural networks\n, weight (\nW\n) and bias (\nB\n) are the parameters that the model learns during training.\nAdjust\nW\nand\nB\nso that\nŷ\nis as close as possible to\ny\nfor all training examples.\nProcess\nStart with\nrandom values\nfor\nW and B\n.\nCalculate the\nloss\nand\ncost\nfor the current parameters.\nUse an\noptimization algorithm\n(e.g.,\ngradient descent\n) to update\nW and B\n.\nOne way to do this is with an algorithm called\ngradient descent\n. Gradient descent helps us find the best weights and biases step by step.\nConclusion\nLoss functions are very important for training neural networks. They tell us how wrong our network is. We use this information to improve the network's predictions. By choosing the right loss function and using algorithms like gradient descent, we can train networks to solve complex problems.\nReady to dive deeper?\nContinue exploring the world of neural networks -\nDeep Learning Essential\n#\nneural-networks\n#\nloss-functions\n#\ndeep-learning\n#\nconcept\n#\ntutorial\n57\nviews", "metadata": { "title": "5. Understanding Loss Functions in Neural Networks", "description": "Learn about loss functions in neural networks, their role in training, and how they help optimize predictions with examples and techniques", "keywords": null, "author": null, "og:title": "5. Understanding Loss Functions in Neural Networks", "og:description": "Learn about loss functions in neural networks, their role in training, and how they help optimize predictions with examples and techniques", "og:url": "https://mfahadbashir.hashnode.dev/5-understanding-loss-functions-in-neural-networks", "og:site_name": "Muhammad Fahad Bashir", "og:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743474274314/2e7a5489-92cc-4f4d-bdb3-cae336cf0099.png", "og:image:width": "1200", "og:image:height": "630", "og:type": "article", "twitter:card": "summary_large_image", "twitter:title": "5. Understanding Loss Functions in Neural Networks", "twitter:description": "Learn about loss functions in neural networks, their role in training, and how they help optimize predictions with examples and techniques", "twitter:image": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743474274314/2e7a5489-92cc-4f4d-bdb3-cae336cf0099.png", "article:published_time": "2025-04-01T02:25:51.366Z", "article:modified_time": "2025-04-01T02:27:46.687Z", "article:author": "https://mfahadbashir.hashnode.dev/@mfahadbashir" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [], "images": [ { "status": "ok", "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743474274314/2e7a5489-92cc-4f4d-bdb3-cae336cf0099.png", "bytes": 775535, "reused": true, "local_path": "attachments/3629d1909f0b/images/000.png", "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743474274314/2e7a5489-92cc-4f4d-bdb3-cae336cf0099.png", "alt": "5. Understanding Loss Functions in Neural Networks", "downloaded": true }, { "status": "ok", "url": "https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/30234805/What-are-neural-networks-Banner.svg", "bytes": 9149, "reused": false, "local_path": "attachments/3629d1909f0b/images/body-e9e039c0222938eb077a.svg", "src": "https://d3lkc3n5th01x7.cloudfront.net/wp-content/uploads/2023/05/30234805/What-are-neural-networks-Banner.svg", "alt": "Neural networks: Architecture, applications, case studies, development and implementation", "downloaded": true }, { "status": "ok", "url": "https://assets.holloway.com/editorial/images/making-things-think/10.9.png", "bytes": 651737, "reused": true, "local_path": "attachments/3629d1909f0b/images/003.png", "src": "https://assets.holloway.com/editorial/images/making-things-think/10.9.png", "alt": "A Brief Overview of Deep Learning — Making Things Think: How AI and Deep Learning Power the Products We Use", "downloaded": true }, { "status": "ok", "url": "https://miro.medium.com/v2/resize:fit:1400/1*hC94sOK3bB4junpyRMnXwg.png", "bytes": 324684, "reused": true, "local_path": "attachments/3629d1909f0b/images/004.png", "src": "https://miro.medium.com/v2/resize:fit:1400/1*hC94sOK3bB4junpyRMnXwg.png", "alt": "Understanding Loss Functions for Classification | by Nghi Huynh | Medium", "downloaded": true }, { "status": "ok", "url": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743445148148/913f858f-8317-4b7d-91ef-966fde4e8b0b.png", "bytes": 44289, "reused": true, "local_path": "attachments/3629d1909f0b/images/005.png", "src": "https://cdn.hashnode.com/res/hashnode/image/upload/v1743445148148/913f858f-8317-4b7d-91ef-966fde4e8b0b.png", "alt": "", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://mfahadbashir.hashnode.dev/5-understanding-loss-functions-in-neural-networks.md", "label": "View as Markdown", "type": "a" }, { "href": "https://hashnode.com/@mfahadbashir", "label": "M", "type": "a" }, { "href": "https://www.linkedin.com/in/muhammad-fahad-bashir-09568216a/", "label": "", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/series/deep-learning-essentials", "label": "Part of series Deep Learning Essentials", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/tag/neural-networks", "label": "# neural-networks", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/tag/loss-functions", "label": "# loss-functions", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/tag/deep-learning", "label": "# deep-learning", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/tag/concept", "label": "# concept", "type": "a" }, { "href": "https://mfahadbashir.hashnode.dev/tag/tutorial", "label": "# tutorial", "type": "a" } ], "retry_at": "2026-10-03T08:07:54", "repair_version": 2, "repaired_at": "2026-10-14T10:50:43Z", "source_url": "https://mfahadbashir.hashnode.dev/5-understanding-loss-functions-in-neural-networks", "papers": [], "actual_video_files": [], "body_chars": 5345, "expected_body_images": 5, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/3629d1909f0b/pages/source.html", "local_html": "attachments/3629d1909f0b/pages/article.html", "body_base_path": "attachments/3629d1909f0b/pages/", "body_markdown_path": "attachments/3629d1909f0b/pages/article.md" }