{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Tharun",
  "title_listed": "The Cost Function: How Wrong Are We, Exactly?",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://tharunpandya.hashnode.dev/the-cost-function-how-wrong-are-we-exactly",
  "publish_date": "2026-08-12",
  "corpus_stage": "",
  "notes": "",
  "id": "eb4e4d0a9508",
  "collected_at": "2026-10-02T14:10:45",
  "title": "The Cost Function: How Wrong Are We, Exactly?",
  "body_markdown": "The Cost Function: How Wrong Are We, Exactly?\nUpdated\nAugust 12, 2026\n•\n8\nmin read\n•\nView as Markdown\nT\nTharun\nLast week, the delivery app on my phone promised my order in 32 minutes. It took 46. I wasn't really annoyed about the food being late. I was annoyed that the model behind that estimate had no way of knowing it was wrong until the food actually showed up at my door.\nIn the last post, we found w = 5 and b = 7 by eyeballing a clean little table, and that pair predicted delivery times almost perfectly. But that raises an obvious question: if someone else had guessed w = 4 and b = 10, how would we know which guess was better without just staring at both and going with our gut?\nWe need a way to turn \"how wrong is this guess\" into an actual number. That number has a name in machine learning. It is called the\ncost function\n. This post is about building that number from scratch, by hand first, then in code.\nMeasuring \"Wrongness\" for a Single Guess\nHere is the same delivery table from the last post.\nDistance (km)\nActual Delivery Time (min)\n1\n12\n2\n17\n3\n22\n4\n27\n5\n32\nWe already know w = 5, b = 7 fits this table perfectly. But suppose we did not know that, and we guessed w = 4, b = 10 instead. Here is what that guess predicts for each row, and how far off it is from the actual value.\nDistance\nPrediction\nActual\nError\n1\n14\n12\n2\n2\n18\n17\n1\n3\n22\n22\n0\n4\n26\n27\n−1\n5\n30\n32\n−2\nNow we have five errors: 2, 1, 0, −1, and −2. The question is how we combine them into one single number that tells us how good or bad this guess is overall.\nThe obvious first move is to just add them up: 2 + 1 + 0 + (−1) + (−2) = 0. That gives us zero, even though the predictions are clearly not perfect. The positive and negative errors simply cancel each other out.\nWhy We Square the Errors\nSo instead of adding the errors themselves, we square them.\n2² = 4\n1² = 1\n0² = 0\n(−1)² = 1\n(−2)² = 4\nNow none of the errors can cancel each other out, since squaring a negative number gives the same positive result as squaring its positive counterpart.\nSquaring also makes bigger mistakes matter more. An error of 1 contributes 1, while an error of 5 contributes 25.\nAdd these squared errors up: 4 + 1 + 0 + 1 + 4 = 10.\nWe have 5 examples, so dividing by 5 gives us the mean squared error. We then include an extra factor of 1/2. That factor does not change which guess is better, but it makes the derivatives cleaner when we get to gradient descent.\ncost = 10 ÷ (2 × 5) = 10 ÷ 10 = 1.0\nThat single number, 1.0, is the cost for the guess w = 4, b = 10.\nThink of the cost as a score for a line. A perfect line gets a score of 0. Small mistakes give a small score. Large mistakes push the score up quickly. So instead of looking at five predictions and trying to judge them yourself, we have reduced the whole table to one number.\nNow the cost for our original guess, w = 5 and b = 7, which we already know fits the table perfectly. Every prediction matches the actual value exactly, so every error is 0, every squared error is 0, and the total is 0 ÷ 10 =\n0\n.\nOur cost is 1.0 for the wrong guess and 0 for the right one. That number does not have a universal meaning by itself; it depends on the scale of the dataset. What matters is that, on the same dataset, a lower cost means a better fit. In this example, a cost of 1 corresponds to an RMSE of about 1.4 minutes, so the errors are relatively small compared with delivery times of 12 to 32 minutes.\nWriting This Down as a Formula\nWe just described the cost function step by step. Now let's compress those steps into one formula.\nJ(w, b) = (1 / 2m) * sum of (prediction − actual) ** 2\nEach piece of that formula maps directly onto something we already did by hand.\nwx + b\nis the prediction for one example.\n(wx + b) − y\nis the error for that example.\n((wx + b) − y) ** 2\nis the squared error.\nThe\nsum\nadds up every squared error across all m examples.\n1/(2m)\nscales that total down, the same way dividing by 5 and then by 2 did above.\nm\nis the number of examples we have, which is 5 in our table, and\nJ(w, b)\nis the name for this whole cost. It depends on which w and b you plug in, which is why we write it as J(w, b) instead of just J. This is the standard cost function used for linear regression.\nTurning This Into Code\nHere is a function that computes this cost, first the plain way, then the vectorized way, the same as we did with the prediction formula last time.\ndef cost_fun(x_list, y_list, w, b):\n\n    m = len(x_list)\n    total_err = 0\n\n    for i in range(m):\n        pred = w * x_list[i] + b\n        err = pred - y_list[i]\n        total_err += err ** 2\n\n    return total_err / (2 * m)\n\n\ndistances = [1, 2, 3, 4, 5]\nactual_times = [12, 17, 22, 27, 32]\n\nwrong_guess = cost_fun(distances, actual_times, w=4, b=10)\ncorrect_guess = cost_fun(distances, actual_times, w=5, b=7)\n\nprint(\"Cost for w=4, b=10:\", wrong_guess)\nprint(\"Cost for w=5, b=7:\", correct_guess)\n\n# Cost for w=4, b=10: 1.0\n# Cost for w=5, b=7: 0.0\nEach line maps back to the formula directly.\npred = w * x_list[i] + b\nis y_prediction = wx + b, the model itself.\nerr = pred - y_list[i]\nis y_prediction − y, the error for that one example.\ntotal_err += err ** 2\nadds up the squared errors.\nreturn total_err / (2 * m)\napplies the 1/(2m) scaling and gives us the cost.\nThat matches exactly what we worked out by hand. Now the vectorized version, using NumPy instead of a loop.\nimport numpy as np\n\ndef cost_fun_vectorized(x, y, w, b):\n    m = len(x)\n    pred = w * x + b\n    err = pred - y\n    return np.sum(err ** 2) / (2 * m)\n\n\ndistances_np = np.array([1, 2, 3, 4, 5])\nactual_times_np = np.array([12, 17, 22, 27, 32])\n\nprint(\"Cost for w=4, b=10:\", cost_fun_vectorized(distances_np, actual_times_np, w=4, b=10))\nprint(\"Cost for w=5, b=7:\", cost_fun_vectorized(distances_np, actual_times_np, w=5, b=7))\n\n# Cost for w=4, b=10: 1.0\n# Cost for w=5, b=7: 0.0\nThere is no loop anywhere in the second version.\nw * x + b\ncomputes every prediction at once,\npred - y\ncomputes every error at once, and\nnp.sum(...)\nadds them all up in one step. The important thing is that the mathematics did not change. We only changed how we wrote the computation, and it stays fast once your dataset stops being five rows and starts being five hundred thousand.\nWhat the Cost Function Still Cannot Do\nWe now have a way to measure exactly how bad a guess is. Given any w and b, we can compute a single number that tells us how well that line fits our data. We could, in theory, try a thousand different combinations of w and b, compute the cost for each one, and keep whichever combination scores lowest.\nThat approach falls apart quickly in practice. With one input feature you might get away with guessing enough combinations by brute force. Real models often have hundreds or thousands of these w-like numbers at once, and randomly guessing your way to the best combination of all of them would take longer than anyone could wait.\nWhat we need instead is a way for the model to look at the cost and figure out which direction to adjust w and b in to make that cost smaller, then repeat that adjustment automatically until the cost stops improving.\nWe can now tell which line is worse. The harder problem is getting from a bad line to a better one without trying thousands of guesses. That is exactly where gradient descent comes in.\nThe full code from this post is up on GitHub here:\nhttps://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Cost_Function_Of_Linear_Regression.ipynb\n#\ncost-function\n#\nlinearregression\n#\nmachine-learning\n#\nai\n#\nartificial-intelligence\n#\npython\n11\nviews",
  "metadata": {
    "title": "The Cost Function: How Wrong Are We, Exactly?",
    "description": "Learn how machine learning measures prediction errors using the cost function, with linear regression examples explained from scratch in Python.",
    "keywords": null,
    "author": null,
    "og:title": "The Cost Function: How Wrong Are We, Exactly?",
    "og:description": "Learn how machine learning measures prediction errors using the cost function, with linear regression examples explained from scratch in Python.",
    "og:url": "https://tharunpandya.hashnode.dev/the-cost-function-how-wrong-are-we-exactly",
    "og:site_name": "Machine Learing From scratch",
    "og:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/27502659-46c3-417d-b956-b964b0b1c66f.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "The Cost Function: How Wrong Are We, Exactly?",
    "twitter:description": "Learn how machine learning measures prediction errors using the cost function, with linear regression examples explained from scratch in Python.",
    "twitter:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/27502659-46c3-417d-b956-b964b0b1c66f.png",
    "article:published_time": "2026-08-12T15:56:55.702Z",
    "article:modified_time": "2026-08-12T16:00:16.833Z",
    "article:author": "https://tharunpandya.hashnode.dev/@tharun-pandya"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/eb4e4d0a9508/code/body-000.txt",
      "chars": 41,
      "language": "text"
    },
    {
      "file": "attachments/eb4e4d0a9508/code/body-001.txt",
      "chars": 36,
      "language": "text"
    },
    {
      "file": "attachments/eb4e4d0a9508/code/body-002.txt",
      "chars": 549,
      "language": "text"
    },
    {
      "file": "attachments/eb4e4d0a9508/code/body-003.txt",
      "chars": 477,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/f3c12634-7df3-47d9-8425-83818d6d4124.png",
      "bytes": 1519444,
      "reused": true,
      "local_path": "attachments/eb4e4d0a9508/images/002.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/f3c12634-7df3-47d9-8425-83818d6d4124.png",
      "alt": "The Cost Function: How Wrong Are We, Exactly?",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://tharunpandya.hashnode.dev/the-cost-function-how-wrong-are-we-exactly.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@tharun-pandya",
      "label": "T",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/tharun-pandya-kodi",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Cost_Function_Of_Linear_Regression.ipynb",
      "label": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Cost_Function_Of_Linear_Regression.ipynb",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/cost-function",
      "label": "# cost-function",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/linearregression",
      "label": "# linearregression",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/ai",
      "label": "# ai",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:04:53",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:29Z",
  "source_url": "https://tharunpandya.hashnode.dev/the-cost-function-how-wrong-are-we-exactly",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 7578,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/eb4e4d0a9508/pages/source.html",
  "local_html": "attachments/eb4e4d0a9508/pages/article.html",
  "body_base_path": "attachments/eb4e4d0a9508/pages/",
  "body_markdown_path": "attachments/eb4e4d0a9508/pages/article.md"
}