{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Tharun",
  "title_listed": "Gradient Descent: Walking Downhill Blindfolded",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://tharunpandya.hashnode.dev/gradient-descent-walking-downhill-blindfolded",
  "publish_date": "2026-08-13",
  "corpus_stage": "",
  "notes": "",
  "id": "33cb808fb089",
  "collected_at": "2026-10-02T14:10:44",
  "title": "Gradient Descent Explained: Walking Downhill Blindfolded",
  "body_markdown": "Gradient Descent: Walking Downhill Blindfolded\nUpdated\nAugust 13, 2026\n•\n10\nmin read\n•\nView as Markdown\nT\nTharun\nIn the last post, we built a way to measure exactly how wrong a guess is, using the cost function. Given any w and b, we can now compute a single number that tells us how badly that line fits our data.\nBut there's a problem we quietly ignored. Knowing that a guess is wrong doesn't tell you which way to fix it. If your cost is 8.5, should you increase w or decrease it? Should b go up or down? The cost function tells you how bad things are. It says nothing about what to do next.\nThis post is about solving exactly that problem.\nImagine You're Standing on a Hill, Blindfolded\nPicture this. You're standing somewhere on a hilly landscape, and your goal is to reach the lowest point in the valley. There's just one catch. You're blindfolded. You can't see the landscape at all.\nWhat would you actually do? Most people would do something like this. Feel the ground under your feet. Figure out which direction feels like it's sloping downward. Take a small step in that direction. Then stop, feel the ground again, and repeat.\nYou wouldn't take one giant leap and hope for the best. A blindfolded giant leap could easily send you flying past the valley and up the opposite hill. Small, careful steps, checked again and again, is how you'd actually get down safely.\nThis is almost exactly what gradient descent does. Except instead of a hill made of dirt and grass, the landscape is the cost function we built in the last post. And instead of your feet feeling the slope, the model calculates it directly.\nConnecting This Back to the Cost Function\nRemember the cost function J(w, b) from the last post. If you tried every possible combination of w and b and plotted the cost for each one, you'd get a bowl shaped surface. Bad guesses for w and b sit high up on the sides of the bowl, where the cost is large. The best possible guess sits right at the bottom, where the cost is at its lowest.\nTraining a model is really just this. Starting somewhere on that bowl, usually a pretty bad, random spot, and walking downhill, step by step, until you reach the bottom.\nThe only thing we need to figure out is: at any given spot on this bowl, which direction is downhill?\nFeeling the Slope, Without Feeling Anything\nIn real hills, you feel the slope with your feet. In gradient descent, the model calculates something called the gradient, which is just a fancy word for slope, at the current w and b.\nThe gradient tells you two things at once. It tells you which direction is uphill, and it tells you how steep the slope is at that point. Since we want to go downhill, not uphill, we move in the opposite direction of the gradient.\nHere's the update rule that does this, written out for both w and b:\nnew w\n= old w − (learning rate × slope of the cost with respect to w)\nnew b\n= old b − (learning rate × slope of the cost with respect to b)\nLet's slow down and unpack every piece of this in plain words.\nThe slope of the cost with respect to w tells us, roughly, \"if I nudge w up a little, does the cost go up or down, and by how much.\"\nA\npositive slope\nmeans increasing w would make things worse, so we should decrease w instead.\nA\nnegative slope\nmeans increasing w would actually help, so we should increase it.\nSubtracting the slope automatically does the right thing in both cases, without us needing to think about which direction to move in by hand.\nThe learning rate, usually written as the Greek letter alpha, controls how big a step we take each time. This is the one number in the whole formula that we choose ourselves, rather than something the model calculates.\nChoosing How Big a Step to Take\nThis is where our earlier tea stall example fits perfectly.\nRemember the stall owner from the first post in this series. Every day, customers told him whether the tea was too sweet or not sweet enough, and he adjusted the sugar he used the next time. Now imagine two extreme versions of him.\nVersion one hears \"too sweet\" and immediately dumps in a huge amount of extra sugar the next time, or panics and removes almost all of it. Every single day, the tea swings wildly between way too sweet and way too plain, and it never actually settles anywhere close to right. This is what happens when the learning rate is too large. The steps are so big that instead of settling near the bottom of the bowl, the model keeps overshooting it, sometimes even ending up worse off than where it started.\nVersion two hears \"too sweet\" and adjusts the sugar by an amount so tiny it's barely noticeable, one grain at a time. Technically, this is moving in the right direction. But it would take him months, maybe years, to actually reach the right amount of sweetness this way. This is what happens when the learning rate is too small. The model does eventually get closer to the answer, but it takes an enormous number of steps to get there, wasting time and computing power along the way.\nThe real stall owner, without ever thinking about it in these terms, found a middle ground. Noticeable adjustments, but not wild swings. That middle ground is exactly what a well chosen learning rate looks like.\nLet's Actually Work Through One Step, By Hand\nLet's go back to our familiar delivery time table.\nDistance (km)\nActual Delivery Time (min)\n1\n12\n2\n17\n3\n22\n4\n27\n5\n32\nWe already know the best answer is w = 5, b = 7. But let's pretend, once more, that we don't know this, and start from the worst possible guess. Let's start completely blindfolded, at w = 0 and b = 0.\nWith w = 0 and b = 0, every single prediction comes out to 0, no matter the distance. So our errors, calculated as prediction minus actual, are simply the negative of each actual value.\nDistance\nPrediction\nActual\nError (prediction minus actual)\n1\n0\n12\n-12\n2\n0\n17\n-17\n3\n0\n22\n-22\n4\n0\n27\n-27\n5\n0\n32\n-32\nThe slope of the cost with respect to w is calculated as the average of each error multiplied by its corresponding distance:\nslope for w = (1/5) × [(-12 × 1) + (-17 × 2) + (-22 × 3) + (-27 × 4) + (-32 × 5)]\nslope for w = (1/5) × [-12 - 34 - 66 - 108 - 160]\nslope for w = (1/5) × (-380)\nslope for w = -76\nThe slope of the cost with respect to b is simply the average of the errors themselves, no distance multiplied in:\nslope for b = (1/5) × [-12 - 17 - 22 - 27 - 32]\nslope for b = (1/5) × (-110)\nslope for b = -22\nBoth slopes came out negative, which tells us both w and b should increase. That matches what we'd expect, since our guess of 0 and 0 is far too low compared to the correct answer of 5 and 7.\nNow let's pick a learning rate. We'll use alpha equals 0.01, a fairly small, safe value. Applying the update rule:\nnew w = 0 - (0.01 × -76) = 0 + 0.76 = 0.76\nnew b = 0 - (0.01 × -22) = 0 + 0.22 = 0.22\nAfter just one single step, our guess moved from w = 0, b = 0 to w = 0.76, b = 0.22. Still far from the correct w = 5, b = 7, but moving in exactly the right direction.\nThis is the part that might surprise you. That's the entire algorithm. Compute the slope, take a small step against it, and repeat. Do this once, and you barely move. Do this a thousand times, and you land almost exactly on w = 5, b = 7, without ever eyeballing a table again.\nWriting This as Code\nLet's turn this into a function that does the update we just did by hand, and then repeats it automatically, many times.\ndef gradient_descent(x_list, y_list, w, b, learning_rate, iterations):\n    m = len(x_list)\n\n    for _ in range(iterations):\n        slope_w = 0\n        slope_b = 0\n\n        for i in range(m):\n            prediction = w * x_list[i] + b\n            error = prediction - y_list[i]\n            slope_w += error * x_list[i]\n            slope_b += error\n\n        slope_w = slope_w / m\n        slope_b = slope_b / m\n\n        w = w - learning_rate * slope_w\n        b = b - learning_rate * slope_b\n\n    return w, b\n\n\ndistances = [1, 2, 3, 4, 5]\nactual_times = [12, 17, 22, 27, 32]\n\nfinal_w, final_b = gradient_descent(distances, actual_times, w=0, b=0, learning_rate=0.01, iterations=10000)\n\nprint(\"Final w:\", final_w)\nprint(\"Final b:\", final_b)\n\n# Final w: close to 5\n# Final b: close to 7\nNotice we never told this function that the answer was w = 5 and b = 7. It started at 0 and 0, completely blind, and found its own way there, purely by repeating the same small correction over and over.\nHere's the same thing, vectorized with NumPy, since real datasets will have far more than five rows:\nimport numpy as np\n\ndef gradient_descent_vectorized(x, y, w, b, learning_rate, iterations):\n    m = len(x)\n\n    for _ in range(iterations):\n        predictions = w * x + b\n        errors = predictions - y\n\n        slope_w = np.sum(errors * x) / m\n        slope_b = np.sum(errors) / m\n\n        w = w - learning_rate * slope_w\n        b = b - learning_rate * slope_b\n\n    return w, b\n\n\ndistances_np = np.array([1, 2, 3, 4, 5])\nactual_times_np = np.array([12, 17, 22, 27, 32])\n\nfinal_w, final_b = gradient_descent_vectorized(distances_np, actual_times_np, w=0, b=0, learning_rate=0.01, iterations=10000)\n\nprint(\"Final w:\", final_w)\nprint(\"Final b:\", final_b)\nBoth versions do the exact same thing. Start with a bad guess. Look at the slope. Take a small step against it. Repeat, over and over, until the guess settles near the bottom of the bowl.\nWhat We've Actually Built\nTake a moment to notice what just happened across these last three posts. In the second post, we could only predict an outcome if a human had already handed us the right w and b. In this post, the model finds those numbers entirely on its own, using nothing but data and this one repeated idea of stepping downhill.\nThis combination, a prediction formula, a cost function to measure wrongness, and gradient descent to fix that wrongness automatically, is the actual engine behind linear regression. Every more advanced model you'll meet later in this series, no matter how complicated it sounds, is built on some variation of this exact same loop.\nSo far, we've only worked with one input feature, distance. Real problems are rarely that simple. A delivery time doesn't just depend on distance. It depends on the number of items in the order, the time of day, how busy the restaurant is, and plenty more. The next post is about extending everything we just built to handle more than one input at a time.\nThe full code from this post is up on GitHub here:\nhttps://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Gradient_Descent.ipynb\nSee you in the next one.\n#\nmachine-learning\n#\ngradient-descent\n#\nlinearregression\n#\npython\n#\nmathematics\n#\nai\n#\nartificial-intelligence\n23\nviews",
  "metadata": {
    "title": "Gradient Descent Explained: Walking Downhill Blindfolded",
    "description": "gradient descent from scratch using a simple blindfolded downhill analogy, with learning rates, slope calculations, a worked example, and Python code.",
    "keywords": null,
    "author": null,
    "og:title": "Gradient Descent Explained: Walking Downhill Blindfolded",
    "og:description": "gradient descent from scratch using a simple blindfolded downhill analogy, with learning rates, slope calculations, a worked example, and Python code.",
    "og:url": "https://tharunpandya.hashnode.dev/gradient-descent-walking-downhill-blindfolded",
    "og:site_name": "Machine Learing From scratch",
    "og:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/f26a56d9-0f53-4a61-8393-cdbd8f26a4a9.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Gradient Descent Explained: Walking Downhill Blindfolded",
    "twitter:description": "gradient descent from scratch using a simple blindfolded downhill analogy, with learning rates, slope calculations, a worked example, and Python code.",
    "twitter:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/f26a56d9-0f53-4a61-8393-cdbd8f26a4a9.png",
    "article:published_time": "2026-08-13T15:38:16.555Z",
    "article:modified_time": "2026-08-13T15:38:16.555Z",
    "article:author": "https://tharunpandya.hashnode.dev/@tharun-pandya"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/33cb808fb089/code/body-000.txt",
      "chars": 179,
      "language": "text"
    },
    {
      "file": "attachments/33cb808fb089/code/body-001.txt",
      "chars": 95,
      "language": "text"
    },
    {
      "file": "attachments/33cb808fb089/code/body-002.txt",
      "chars": 86,
      "language": "text"
    },
    {
      "file": "attachments/33cb808fb089/code/body-003.txt",
      "chars": 783,
      "language": "text"
    },
    {
      "file": "attachments/33cb808fb089/code/body-004.txt",
      "chars": 656,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/21bbf8bc-2034-4570-88bd-6e2b9fe7aeef.png",
      "bytes": 2018272,
      "reused": true,
      "local_path": "attachments/33cb808fb089/images/002.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/21bbf8bc-2034-4570-88bd-6e2b9fe7aeef.png",
      "alt": "Gradient Descent: Walking Downhill Blindfolded",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://tharunpandya.hashnode.dev/gradient-descent-walking-downhill-blindfolded.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@tharun-pandya",
      "label": "T",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/tharun-pandya-kodi",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Gradient_Descent.ipynb",
      "label": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Gradient_Descent.ipynb",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/gradient-descent",
      "label": "# gradient-descent",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/linearregression",
      "label": "# linearregression",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/mathematics",
      "label": "# mathematics",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/ai",
      "label": "# ai",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:04:50",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:28Z",
  "source_url": "https://tharunpandya.hashnode.dev/gradient-descent-walking-downhill-blindfolded",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 10606,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/33cb808fb089/pages/source.html",
  "local_html": "attachments/33cb808fb089/pages/article.html",
  "body_base_path": "attachments/33cb808fb089/pages/",
  "body_markdown_path": "attachments/33cb808fb089/pages/article.md"
}