{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Tharun",
  "title_listed": "Polynomial Regression: Teaching a Line to Bend",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://tharunpandya.hashnode.dev/polynomial-regression-teaching-a-line-to-bend",
  "publish_date": "2026-08-22",
  "corpus_stage": "",
  "notes": "",
  "id": "8376268113b2",
  "collected_at": "2026-10-02T14:10:44",
  "title": "Polynomial Regression: Teaching a Line to Bend",
  "body_markdown": "Polynomial Regression: Teaching a Line to Bend\nUpdated\nAugust 22, 2026\n•\n10\nmin read\n•\nView as Markdown\nT\nTharun\nTake our best straight-line model so far, the one that fit our five-order table perfectly. Now widen the range a bit. Some orders close by, a couple kilometers away. Some far, ten or twelve kilometers away. Feed all of them to the same straight-line model.\nIt starts missing. Not by a little. By a lot. And the misses aren't random, they follow a pattern. Short orders come in higher than predicted. Long orders come in higher too. Only the orders in the middle land close to the line.\nThat pattern is the whole story of this post.\nA Wider Table\nHere are six orders, spread across a wider range of distances than we've used before.\nDistance (km)\nActual Time (min)\n2\n11\n4\n21\n6\n35\n8\n53\n10\n75\n12\n101\nOne thing worth being upfront about: I built this table on purpose so the numbers follow a clean quadratic pattern. It's a teaching example, not a real delivery log, and real data would be a lot messier. But it's clean for a reason, it lets us see the idea we're after without any noise getting in the way.\nLook at how the time grows as distance grows. From 2 km to 4 km, time goes up by 10 minutes. From 4 km to 6 km, it goes up by 14 minutes. From 6 to 8, it jumps by 18. From 8 to 10, by 22. From 10 to 12, by 26.\nEach step adds more time than the last one did. A straight line can't do that. A straight line adds the exact same amount for every extra kilometer, no matter where you are on it. This data isn't behaving like a straight line at all.\nWhy a Straight Line Struggles Here\nIt's tempting to explain this away with a story: short orders go out on bikes, long orders switch to scooters or cars, get grouped with other stops, pick up delays a short trip never sees. That story might even be true sometimes. But I want to be careful here, because our table wasn't built from any of that. It was built directly from a formula. So the honest version of this section isn't \"delivery time is secretly quadratic.\" It's this:\nReal-world data can bend for all sorts of reasons, traffic, batching, prep time, road conditions, take your pick. A straight line has no way to represent any of that, because it assumes every extra kilometer costs exactly the same amount of time, forever. Our job isn't to prove why this particular curve happens. It's to build a model that isn't stuck assuming everything is a straight line in the first place.\nLet's See the Straight Line Actually Fail\nWe already know how to fit a straight line to this data, we built the exact code for it two posts ago. Let's run it on this new table and look at what comes out.\nw, b = gradient_descent(distances, actual_times, w=0, b=0, learning_rate=0.001, iterations=20000)\nThe line it settles on gives these predictions:\nDistance\nActual\nPredicted by the line\nError\n2\n11\n4.6\n+6.4\n4\n21\n22.5\n−1.5\n6\n35\n40.4\n−5.4\n8\n53\n58.4\n−5.4\n10\n75\n76.3\n−1.3\n12\n101\n94.2\n+6.8\nLook at that error column on its own for a second. It starts positive, dips negative in the middle, then swings positive again at the far end. That shape, high, dip, high, is the model's way of telling us something. The line isn't randomly wrong. It's too low at both ends and too high in the middle, which means whatever pattern it's missing has a curve to it. That's the clue.\nAt 2 km, the line predicts 4.6 minutes when the real answer is 11. At 12 km, it predicts 94.2 when the real answer is 101. It can only pass through the middle of the data and miss both edges.\nGiving the Model a New Number to Work With\nHere's the fix, and it's simpler than it sounds. We don't need a new kind of model. We just need to hand our existing straight-line model a second input, one that already bends.\nThat second input is distance squared, written as x². Look at how it grows compared to plain distance:\nDistance (x)\nDistance Squared (x²)\n2\n4\n4\n16\n6\n36\n8\n64\n10\n100\n12\n144\nDistance itself grows steadily, 2, 4, 6, 8. But x² shoots up faster and faster, 4, 16, 36, 64. That's exactly the shape our delivery times were showing. We're not changing the model's job. We're changing what we give it to look at.\nIf we let the model use both x and x² as inputs, it can multiply x² by some weight and get exactly the curve it needs, using nothing but the multiple linear regression code we already built.\nThe formula barely changes from what we already know:\nprediction = w1 × x + w2 × x² + b\nThis is still, underneath everything, the same linear regression from the earlier posts. We haven't invented a new algorithm. We've just fed it a new number to multiply.\nHere's the part worth sitting with for a second. If it's still linear regression, how is it allowed to draw a curve? The answer is that \"linear\" here describes how the model treats its weights, not how the final prediction looks. The prediction curves with respect to x, because x² bends. But with respect to w1 and w2, it's still just addition and multiplication, the same thing gradient descent has been doing since the first post. We didn't teach it how to draw curves. We gave it a curved input and let the same gradient descent do its job.\nThis step, going from one input to more than one, already happened once before in this series, when we moved from a single feature to multiple. Polynomial regression isn't a third, unrelated idea, it's the same move again, just with x² standing in as a new \"feature\" built from x itself.\nChecking This By Hand\nLet's pick the order at 6 km and check it ourselves. Its new second input is 6² = 36.\nIf w1 = 2, w2 = 0.5, and b = 5, the prediction works out to:\nprediction = 2 × 6 + 0.5 × 36 + 5 \nprediction = 12 + 18 + 5 \nprediction = 35\nThat matches the actual value in our table exactly. And when we train the model on this table, it discovers weights that land right around these same numbers, which is exactly what we'll do next.\nTurning This Into Code\nFirst, let's build the x² column.\nimport numpy as np\n\ndistances = np.array([2, 4, 6, 8, 10, 12], dtype=float)\nactual_times = np.array([11, 21, 35, 53, 75, 101], dtype=float)\n\ndistance_squared = distances ** 2\n\nX = np.column_stack([distances, distance_squared])\nprint(X)\n[[  2.   4.]\n [  4.  16.]\n [  6.  36.]\n [  8.  64.]\n [ 10. 100.]\n [ 12. 144.]]\nNotice the two columns here. One goes up to 12. The other goes up to 144. Now our two inputs are on very different scales, and that's the exact same mismatch we spent the whole last post fixing. So before training, we scale both columns the same way we already know how to.\nX_mean = X.mean(axis=0)\nX_std = X.std(axis=0)\nX_scaled = (X - X_mean) / X_std\nNow we hand this straight to the same\ngradient_descent\nfunction from the multiple regression post, completely unchanged.\nw, b = gradient_descent(X_scaled, actual_times, w=np.zeros(2), b=0.0, learning_rate=0.1, iterations=5000)\n\npredictions = X_scaled.dot(w) + b\nprint(predictions)\n[ 11.  21.  35.  53.  75. 101.]\nEvery single prediction lands right on the actual value. But the predictions alone aren't the interesting part. Let's look at what the model actually learned:\nprint(\"Weights:\", w)\nprint(\"Bias:\", b)\nWeights: [2.  0.5]\nBias: 5.0\nThose numbers should look familiar. They're the same 2 and 0.5 we used in the hand calculation earlier. The model found them on its own, starting from nothing but zeros, using the same gradient descent code we've reused since the second post in this series. Put back into the formula, that's:\nprediction = 2 × x + 0.5 × x² + 5\nWhich, unsurprisingly, is exactly the equation this table was built from in the first place. The only new idea in this entire post was handing the model one extra number to work with.\nThat said, I'd be doing you a disservice if I let \"every prediction is exact\" sit there unchallenged. Six data points, three numbers to learn, and a dataset built precisely from the formula we're asking it to find, of course it lands exactly. That's not really a demonstration of how good the model is. It's a demonstration of how clean the setup is. On real data, a model that fits every single training point perfectly is often a warning sign, not a win, something we'll get into properly when we talk about overfitting.\nThis Trick Doesn't Stop at x²\nOnce you see that x² is just another input, there's nothing stopping you from adding x³, x⁴, or beyond, the same way. Each one lets the line bend a little differently. In practice, you rarely need to go very far past x² or x³ before the model starts fitting noise instead of the real pattern, something we'll come back to properly when we talk about overfitting. But the core idea stops here. A curve, in the end, is just a straight line that's been handed the right extra numbers.\nWhere We Go From Here\nEvery model we've built across this entire series, all the way back to the first post, has been answering the same kind of question. Given some inputs, predict a number. Delivery time in minutes. A number that could be small, large, anywhere on a scale.\nBut not every real question looks like that. Sometimes the answer isn't a number at all. Sometimes it's just yes or no. Will this order be late, or won't it? That's not a number to predict, it's a decision to make. And it turns out, everything we've built so far is about to hit its first real wall trying to answer a question shaped like that.\nThat's exactly where we're headed next.\nThe full code for this post is up on GitHub here:\ngithub.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Polynomial_Regression.ipynb\nSee you in the next one.\n#\nmachine-learning\n#\npolynomialregression\n#\nregression\n#\nlinearregression\n#\nfeature-engineering\n#\ngradient-descent\n#\npython\n#\ndata-science\n#\nai\n#\nartificial-intelligence\n7\nviews",
  "metadata": {
    "title": "Polynomial Regression: Teaching a Line to Bend",
    "description": "Learn polynomial regression from scratch using a delivery-time example, x^2, feature engineering, and gradient descent to fit curved patterns.",
    "keywords": null,
    "author": null,
    "og:title": "Polynomial Regression: Teaching a Line to Bend",
    "og:description": "Learn polynomial regression from scratch using a delivery-time example, x^2, feature engineering, and gradient descent to fit curved patterns.",
    "og:url": "https://tharunpandya.hashnode.dev/polynomial-regression-teaching-a-line-to-bend",
    "og:site_name": "Machine Learing From scratch",
    "og:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/7022e37d-ef83-4505-8bb0-4172420b2eb1.png",
    "og:image:width": "1200",
    "og:image:height": "630",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Polynomial Regression: Teaching a Line to Bend",
    "twitter:description": "Learn polynomial regression from scratch using a delivery-time example, x^2, feature engineering, and gradient descent to fit curved patterns.",
    "twitter:image": "https://cdn.hashnode.com/uploads/og-images/6a3f9a25a837e83b51cc8712/7022e37d-ef83-4505-8bb0-4172420b2eb1.png",
    "article:published_time": "2026-08-22T20:35:25.542Z",
    "article:modified_time": "2026-08-22T20:35:25.542Z",
    "article:author": "https://tharunpandya.hashnode.dev/@tharun-pandya"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/8376268113b2/code/body-000.txt",
      "chars": 98,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-001.txt",
      "chars": 34,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-002.txt",
      "chars": 77,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-003.txt",
      "chars": 236,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-004.txt",
      "chars": 79,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-005.txt",
      "chars": 78,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-006.txt",
      "chars": 160,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-007.txt",
      "chars": 32,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-008.txt",
      "chars": 39,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-009.txt",
      "chars": 29,
      "language": "text"
    },
    {
      "file": "attachments/8376268113b2/code/body-010.txt",
      "chars": 34,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/20e52622-beb9-429c-9bcd-9924ca3f983e.png",
      "bytes": 1774601,
      "reused": true,
      "local_path": "attachments/8376268113b2/images/002.png",
      "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/20e52622-beb9-429c-9bcd-9924ca3f983e.png",
      "alt": "Polynomial Regression: Teaching a Line to Bend",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://tharunpandya.hashnode.dev/polynomial-regression-teaching-a-line-to-bend.md",
      "label": "View as Markdown",
      "type": "a"
    },
    {
      "href": "https://hashnode.com/@tharun-pandya",
      "label": "T",
      "type": "a"
    },
    {
      "href": "https://linkedin.com/in/tharun-pandya-kodi",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Polynomial_Regression.ipynb",
      "label": "github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Polynomial_Regression.ipynb",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/machine-learning",
      "label": "# machine-learning",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/polynomialregression",
      "label": "# polynomialregression",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/regression",
      "label": "# regression",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/linearregression",
      "label": "# linearregression",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/feature-engineering",
      "label": "# feature-engineering",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/gradient-descent",
      "label": "# gradient-descent",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/python",
      "label": "# python",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/data-science",
      "label": "# data-science",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/ai",
      "label": "# ai",
      "type": "a"
    },
    {
      "href": "https://tharunpandya.hashnode.dev/tag/artificial-intelligence",
      "label": "# artificial-intelligence",
      "type": "a"
    }
  ],
  "retry_at": "2026-10-03T08:04:43",
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:26Z",
  "source_url": "https://tharunpandya.hashnode.dev/polynomial-regression-teaching-a-line-to-bend",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 9670,
  "expected_body_images": 1,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/8376268113b2/pages/source.html",
  "local_html": "attachments/8376268113b2/pages/article.html",
  "body_base_path": "attachments/8376268113b2/pages/",
  "body_markdown_path": "attachments/8376268113b2/pages/article.md"
}