{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Tharun",
  "title_listed": "Feature Scaling in Machine Learning: Why It Matters",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://tharunpandya.hashnode.dev/feature-scaling-in-machine-learning-why-it-matters",
  "publish_date": "2026-08-19",
  "corpus_stage": "",
  "notes": "",
  "id": "22b8d76f52f4",
  "collected_at": "2026-10-02T14:10:44",
  "title": "Feature Scaling in Machine Learning: Why It Matters",
  "body_markdown": "Feature Scaling in Machine Learning: Why It Matters\nUpdated\nAugust 19, 2026\n•\n10\nmin read\n•\nView as Markdown\nT\nTharun\nTake the exact same model, the exact same data, the exact same code. Change one thing: write the distance column in meters instead of kilometers. Nothing else moves. Run it, and the model doesn't just learn a little worse. It falls apart completely, spitting out numbers so large your computer can't even hold them, in under 60 steps.\nThat's not a bug. That's what this post is about.\nA quick note first.\nI'd originally planned to show this by adding a fourth input, the price of the order. But the more I sat with it, the less I believed it. There's no honest reason the price someone paid should change how long their food takes to arrive, so it would've been a made-up example standing in for a real problem. The real problem doesn't need a made-up example. It shows up the moment two ordinary numbers happen to live on different scales, which is exactly what changing kilometers to meters does. Same orders, same data, just a different ruler.\nThe Table, With One Number Changed\nHere's our familiar delivery table — the exact same five orders we've used since the multiple linear regression post. Nothing about the deliveries changed. The only thing that changed is how we wrote down the distance: instead of 1, 2, 3, 2, and 4 kilometers, it's now 1000, 2000, 3000, 2000, and 4000 meters. Same real-world trip, same real-world time, just a different ruler.\nDistance (x1, meters)\nItems (x2)\nPeak Hours (x3)\nActual Time (min)\n1000\n2\n0\n16\n2000\n1\n1\n29\n3000\n3\n0\n28\n2000\n4\n1\n35\n4000\n2\n0\n31\nLook at how differently these three columns are dressed. Distance struts around in the thousands. Items barely leaves single digits. Peak hours never wears anything but a 0 or a 1. Same orders as before, same real-world distances, just measured with a different ruler, and already something feels off.\nA Bucket and a Mug\nHere's a small way to feel why that matters, before we touch any code.\nSay you're filling two containers with the same jug, one scoop at a time. One container is a large bucket. The other is a small mug. Pour one scoop into the bucket, and the water barely rises. Pour that same scoop into the mug, and it overflows. The scoop wasn't wrong — the two containers were just never built on the same scale.\nGradient descent runs into exactly this problem. Every post so far, we've talked about it taking a step, sized by something called the learning rate, and adjusting every weight by that same step, every round. One step size, applied to a number in the thousands and a number that's only ever 0 or 1. A step that's sensible for one is going to be wildly wrong for the other, and gradient descent doesn't get to pick a different size for each.\nLet's Actually Watch It Happen\nWe already built a\ngradient_descent\nfunction two posts ago. Let's not touch a single line of it, and just hand it this new table.\nimport numpy as np\n\ndef gradient_descent(X, y, w, b, learning_rate, iterations):\n    m = len(y)\n    for _ in range(iterations):\n        predictions = X.dot(w) + b\n        errors = predictions - y\n        gradient_w = X.T.dot(errors) / m\n        gradient_b = np.sum(errors) / m\n        w = w - learning_rate * gradient_w\n        b = b - learning_rate * gradient_b\n    return w, b\n\nX = np.array([\n    [1000, 2, 0],\n    [2000, 1, 1],\n    [3000, 3, 0],\n    [2000, 4, 1],\n    [4000, 2, 0],\n])\nactual_times = np.array([16, 29, 28, 35, 31])\nLast post, a learning rate of 0.05 worked beautifully. Same rate, same code, new table:\nw, b = gradient_descent(X, actual_times, np.zeros(3), 0.0, learning_rate=0.05, iterations=100)\nBy the 56th step, the weights have already grown too large for the computer to represent as real numbers. Nothing about our code changed. Only the unit distance was measured in did, and that alone was enough to break it.\nSo we shrink the learning rate, over and over, hunting for one small enough to survive. Here's where that hunt lands:\nLearning rate\nWhat happens after 20,000 steps\n0.0000005\nstill breaks, by step 798\n0.00000025\nsurvives, but barely learns anything, stuck at a cost of 41.70\nThat second number is small enough to survive. It's also so small that after 20,000 steps, it still hasn't found a good fit. And here's the part that stings: nudge it up by about a quarter, to 0.00000032, and it breaks again, this time by step 4,301.\nThis is the real cost of leaving your numbers on mismatched scales. It's not just slower training. You get boxed into a learning rate so narrow that even the safest setting barely teaches the model anything.\nThe Fix: Put Every Feature on the Same Ruler\nIf two numbers being on wildly different scales is the whole problem, the fix writes itself: bring every feature onto a similar scale before training even starts. Three common ways to do that.\nMin-Max Scaling\nsqueezes every value into the range 0 to 1.\nx scaled = (x - minimum) / (maximum - minimum)\nMean Normalization\ndoes something similar, but centers the values around 0 instead of starting at 0.\nx scaled = (x - mean) / (maximum - minimum)\nZ-score Standardization\nuses how spread out the data actually is, its standard deviation, instead of just its range.\nx scaled = (x - mean) / standard deviation\nAfter this, a feature ends up with an average of 0 and a standard deviation of 1. It's not the only valid choice, and min-max scaling is common too, but for this post we'll go with Z-score standardization because it pairs nicely with the gradient descent example we're about to rerun.\nIf you picture the cost function as a bowl-shaped surface, the kind we drew back in the cost function post, unscaled data stretches that bowl into a long, narrow oval, squeezed almost flat along one direction. Gradient descent, walking downhill on that shape, keeps overshooting from one steep wall to the other instead of walking straight to the bottom. Scaling reshapes that oval back into something closer to a circle, so downhill actually points toward the answer.\nWorking Through One By Hand\nLet's use our own distance column to see all three in action: 1000, 2000, 3000, 2000, and 4000 meters.\nThe minimum is 1000, the maximum is 4000, the mean is 2400, and the standard deviation works out to about 1019.80.\nLet's scale one value, 3000 m (the third order), using all three formulas.\nMin-Max:   (3000 - 1000) / (4000 - 1000) = 2000 / 3000 = 0.6667\nMean Norm: (3000 - 2400) / (4000 - 1000) = 600 / 3000 = 0.2000\nZ-score:   (3000 - 2400) / 1019.80 = 600 / 1019.80 = 0.5883\nA number that used to sit in the thousands now lands somewhere small, close to items and peak hours. That's exactly what we needed.\nTurning This Into Code\nThe plain Python version of all three:\ndef min_max_scale(values):\n    v_min = min(values)\n    v_max = max(values)\n    return [(v - v_min) / (v_max - v_min) for v in values]\n\ndef mean_normalize(values):\n    v_min = min(values)\n    v_max = max(values)\n    v_mean = sum(values) / len(values)\n    return [(v - v_mean) / (v_max - v_min) for v in values]\n\ndef z_score_scale(values):\n    v_mean = sum(values) / len(values)\n    variance = sum((v - v_mean) ** 2 for v in values) / len(values)\n    v_std = variance ** 0.5\n    return [(v - v_mean) / v_std for v in values]\n\ndistance_m = [1000, 2000, 3000, 2000, 4000]\n\nprint(min_max_scale(distance_m))\nprint(mean_normalize(distance_m))\nprint(z_score_scale(distance_m))\nAnd the NumPy version, which scales a whole table in one line:\ndef z_score_scale_np(x):\n    return (x - x.mean(axis=0)) / x.std(axis=0)\n\nX_scaled = z_score_scale_np(X.astype(float))\nX\nis a full table now, not a single column, so\nx.mean(axis=0)\nand\nx.std(axis=0)\nwork out a separate mean and spread for each column on its own. Distance gets measured against distance's own spread, items against items', and so on. Our 3000 m example above matches this exactly.\nRetraining, Now Scaled\nSame unchanged\ngradient_descent\nfunction. This time, a learning rate of 0.1, which the unscaled table couldn't have survived at any size.\nw, b = gradient_descent(X_scaled, actual_times, np.zeros(3), 0.0, learning_rate=0.1, iterations=500)\nHere's how the walk went, checked along the way:\nSteps\nw1\nw2\nw3\nb\nCost\n20\n3.897\n2.014\n3.718\n24.420\n8.171\n60\n5.025\n2.059\n4.824\n27.750\n0.005855\n100\n5.094\n2.041\n4.894\n27.799\n0.0000187\n200\n5.099\n2.040\n4.899\n27.800\n≈0\n500\n5.099\n2.040\n4.899\n27.800\n≈0\nTwo things worth pausing on.\nThe bias b lands on 27.8 almost right away and never moves again. That's the plain average of our five delivery times, (16+29+28+35+31)/5. Once every input sits centered around 0, all b has left to represent is the average outcome, and gradient descent finds that almost instantly.\nAnd notice the learning rate itself. We can push it all the way to 1.5 and this still learns cleanly. Only past that does it start to break. Compare that to the unscaled version, which broke the moment we nudged past 0.00000032. On this particular dataset, scaling didn't just make training faster, it let us use a learning rate roughly six million times larger, on the exact same orders. That ratio isn't a universal law of scaling — it's just how much room this one dataset had been wasting.\nOne more thing: these scaled weights aren't readable anymore as \"minutes per kilometer.\" They're attached to standardized versions of the inputs now, not the raw ones. But undo the standardization, and they land right back on w1 = 5, w2 = 2, w3 = 10, the exact numbers our very first multi-input model found by hand, a couple of posts ago, just reached this time in a few hundred steps instead of ten thousand.\nOne Small Thing to Remember\nOnce you scale your training data, any new order you want a prediction for has to be scaled the same way, using the same minimum, maximum, mean, and standard deviation you calculated from training, never freshly calculated ones. Say a new order comes in at 6000 m, 3 items, off-peak. You'd scale that 6000 against our training mean of 2400 and standard deviation of 1019.80, landing at about 3.530, not against whatever a brand new batch of orders happens to average out to. Otherwise you'd be comparing a number measured on one ruler to weights that were learned using a completely different one.\nWhere We Go From Here\nOne small unit change, kilometers to meters, was enough to take a learning rate that worked perfectly and make it useless. Three formulas you could compute by hand fixed it.\nBut every model we've built so far, all the way back to the very first post, has assumed the answer is a straight line. What happens when the pattern in the data genuinely isn't straight at all, when a very long delivery might not even use the same kind of vehicle as a short one? That's next.\nFull code for this post:\nhttps://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Feature_Scaling.ipynb\nSee you in the next one.\n#\nmachine-learning\n#\nfeature-scaling\n#\ngradient-descent\n#\ndata-science\n#\npython\n9\nviews",
  "metadata": {
    "title": "Feature Scaling in Machine Learning: Why It Matters",
    "description": "Learn why feature scaling matters in machine learning and how Min-Max, Mean Normalization, and Z-score scaling help gradient descent learn better.",
    "keywords": null,
    "author": null,
    "og:title": "Feature Scaling in Machine Learning: Why It Matters",
    "og:description": "Learn why feature scaling matters in machine learning and how Min-Max, Mean Normalization, and Z-score scaling help gradient descent learn better.",
    "og:url": "https://tharunpandya.hashnode.dev/feature-scaling-in-machine-learning-why-it-matters",
    "og:site_name": "Machine Learing From scratch",
    "og:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/986ff175-ec4d-4c20-9ff3-5c05e95f5d48.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Feature Scaling in Machine Learning: Why It Matters",
    "twitter:description": "Learn why feature scaling matters in machine learning and how Min-Max, Mean Normalization, and Z-score scaling help gradient descent learn better.",
    "twitter:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/986ff175-ec4d-4c20-9ff3-5c05e95f5d48.png",
    "article:published_time": "2026-08-19T06:55:40.373Z",
    "article:modified_time": "2026-08-19T06:55:40.373Z",
    "article:author": "https://tharunpandya.hashnode.dev/@tharun-pandya"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/22b8d76f52f4/code/body-000.txt",
      "chars": 534,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-001.txt",
      "chars": 95,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-002.txt",
      "chars": 47,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-003.txt",
      "chars": 44,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-004.txt",
      "chars": 43,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-005.txt",
      "chars": 187,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-006.txt",
      "chars": 669,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-007.txt",
      "chars": 119,
      "language": "text"
    },
    {
      "file": "attachments/22b8d76f52f4/code/body-008.txt",
      "chars": 101,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/2706fb6d-344a-4a5f-8963-0ba3f305ace0.png",
      "bytes": 1778884,
      "reused": true,
      "local_path": "attachments/22b8d76f52f4/images/002.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/2706fb6d-344a-4a5f-8963-0ba3f305ace0.png",
      "alt": "Feature Scaling in Machine Learning: Why It Matters",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/843cd656-828c-4abc-8354-87412a90a228.png",
      "bytes": 530199,
      "reused": true,
      "local_path": "attachments/22b8d76f52f4/images/003.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/843cd656-828c-4abc-8354-87412a90a228.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://tharunpandya.hashnode.dev/feature-scaling-in-machine-learning-why-it-matters.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@tharun-pandya",
      "label": "T",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/tharun-pandya-kodi",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Feature_Scaling.ipynb",
      "label": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Feature_Scaling.ipynb",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/feature-scaling",
      "label": "# feature-scaling",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/gradient-descent",
      "label": "# gradient-descent",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/data-science",
      "label": "# data-science",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:04:45",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:27Z",
  "source_url": "https://tharunpandya.hashnode.dev/feature-scaling-in-machine-learning-why-it-matters",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 10906,
  "expected_body_images": 2,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/22b8d76f52f4/pages/source.html",
  "local_html": "attachments/22b8d76f52f4/pages/article.html",
  "body_base_path": "attachments/22b8d76f52f4/pages/",
  "body_markdown_path": "attachments/22b8d76f52f4/pages/article.md"
}