{ "source_file": "0930", "source_platform": "Hashnode", "author": "Tharun", "title_listed": "From Lines to Decisions: Why Regression Fails at Classification", "keywords": "", "domain": "机器学习", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://tharunpandya.hashnode.dev/from-lines-to-decisions-why-regression-fails-at-classification", "publish_date": "2026-08-31", "corpus_stage": "", "notes": "", "id": "9952ce2ffbf4", "collected_at": "2026-10-02T14:10:44", "title": "Why Linear Regression Fails for Classification", "body_markdown": "From Lines to Decisions: Why Regression Fails at Classification\nUpdated\nAugust 31, 2026\n•\n9\nmin read\n•\nView as Markdown\nT\nTharun\nLast time, right at the end, I asked a question and then walked away from it. Will this order be late, or won't it? The final answer isn't a number on a continuous scale. It's a class: yes or no.\nEvery model we've built so far in this series has answered a different kind of question — how many minutes, how much rent, how many rupees. All numbers that can sit anywhere on a scale.\n\"Will it be late\" doesn't have a continuous answer like that. The final class is either yes or no. But as we'll see, the model can still use a number to express how likely each class is.\nSo here's what I want to do in this post: take the model we already trust, the straight line, and try it on this new kind of question. Not because I expect it to work. Because watching it fail, and seeing exactly how it fails, is the fastest way to understand why we need something new.\nThe Same Orders, Two Different Questions\nGo back to the delivery orders we've been using this whole series. Distance from the restaurant, and how long the order actually took.\nDistance (km)\nTime Taken (min)\n1\n14\n3\n22\n5\n30\n7\n41\n9\n47\n11\n55\n\"Predict the time\" is a regression question. That's what we've been doing since the first post. Give the model a distance, get back a number of minutes.\nNow let's ask something else about these exact same orders. Say anything over 40 minutes counts as late. Look at the same table again, but this time with a late column added.\nDistance (km)\nTime Taken (min)\nLate?\n1\n14\nNo\n3\n22\nNo\n5\n30\nNo\n7\n41\nYes\n9\n47\nYes\n11\n55\nYes\nSame orders. Same distances. Completely different question. \"How long will it take\" wants a number. \"Will it be late\" wants a decision. One output is continuous, it can be 14.2 minutes or 39.9 minutes, anything at all. The other output only ever has two possible values. Late, or not late. There's nothing in between.\nOne honest note before we go further: this little dataset is unusually clean. Every short order comes back not late, every long order comes back late, no exceptions. Real delivery data wouldn't be this polite — two orders at the same distance could land on opposite sides because of traffic, a slow kitchen, or a bad light. I'm keeping it this clean on purpose, because right now I want to isolate one problem: why a straight line makes a bad probability model. Messier data can come later.\nThat difference sounds small when you write it in a sentence. It is not small to a model.\nLet's Just Try a Line Anyway\nHere's the thing about \"no\" and \"yes.\" A computer doesn't understand words like that. So before we can do anything, we turn them into numbers. No becomes 0. Yes becomes 1.\nDistance (km)\nLate (0 = No, 1 = Yes)\n1\n0\n3\n0\n5\n0\n7\n1\n9\n1\n11\n1\nNow look at that table honestly for a second. It's still a distance column and a number column. Nothing about it looks different from any of the tables we've fit a straight line to before. So why not just do exactly that? Feed it into the same gradient descent code, and see what happens.\nw, b = gradient_descent(distances, late, learning_rate=0.01, iterations=20000)\nprint(w, b)\nw: 0.1286\nb: -0.2714\nNothing exploded. No error. The model trained just fine — it found a line that minimizes the same squared-error cost we've been using this whole series. Let's look at what it predicts for each order.\nDistance\nActual (0 or 1)\nLine's Prediction\n1\n0\n−0.143\n3\n0\n0.114\n5\n0\n0.371\n7\n1\n0.629\n9\n1\n0.886\n11\n1\n1.143\nBefore trusting that table, let's check one row by hand, the way we've done in earlier posts. Take the order at 7 km:\nprediction = 0.1286(7) - 0.2714 \n prediction = 0.9002 - 0.2714 \n prediction = 0.6288\nThe actual label is 1. The line gives us 0.629. If we pretend that number is a probability, that's about 63% — not a crazy number on its own. And that's the tricky part: 0.629 looks perfectly reasonable as a probability, but looking reasonable isn't the same as actually being one. The trouble shows up at the edges.\nStop on that first row for a second. The actual answer is 0. Not \"close to zero.\" Just zero, flat, that's the whole answer. And the line predicts −0.143.\nWe could try to read that as a probability, but that's exactly where things fall apart. What does negative 0.143 mean here? Negative fourteen percent chance of being late? There's no such thing as a negative chance. You can't be less than 0% sure of anything. The number the line gave us doesn't correspond to any real answer in the world.\nNow look at the last row. Actual answer is 1. The line predicts 1.143 — treat that as a probability and it's saying 114.3% chance of being late. That's not just wrong, it's wrong in a way that shouldn't even be possible. The model has somehow become more than certain. Certainty tops out at 100%.\nWhy This Isn't Just a Rounding Problem\nMy first instinct, the first time I saw this, was to think: fine, just round it. Anything below 0.5 becomes \"not late,\" anything above becomes \"late.\" Problem solved, right?\nWorth pausing on that 0.5 for a second — it isn't something the model learned. It's a threshold we chose, a rule for turning a continuous number into a yes-or-no decision. It'll matter a lot more once we get to the next model.\nRounding does patch the output. But it doesn't fix what's actually broken. Try feeding the model a distance a bit further out than anything it saw during training.\nfor d in [13, 15]:\n print(d, w*d + b)\n13 1.4\n15 1.657\nAt 15 km, the line is telling us the order is 165.7% likely to be late. Rounding will happily turn that into \"late,\" and get the direction right. But it's getting the right answer by accident, using a number that never meant anything in the first place.\nAnd here's the part that actually matters. This isn't a bug we can fix by tweaking the learning rate, training longer, or picking a better starting point — it's baked into what a straight line is. A line has no ceiling and no floor. Give it enough distance in either direction, and it will keep climbing or keep falling, forever. That's just what a line is.\nBut \"chance of being late\" isn't allowed to do that. It has a hard floor at 0 and a hard ceiling at 1, because that's what probability means. Zero percent is the least sure you can ever be. A hundred percent is the most. A tool that can't stay inside those two walls was never going to be the right tool for a question shaped like this one.\nChecking This With a Picture in Your Head\nPicture the six points from our table plotted out. Three sitting flat on the 0 line for the shorter distances. Three sitting flat on the 1 line for the longer ones. Somewhere in between, a jump.\nNow picture our straight line drawn through that same plot. It slides up gently, left to right, the way every line we've trained in this series has. On the left side of the picture, it dips below the 0 line, into negative numbers. On the right side, it rises above the 1 line, into numbers bigger than one.\nThat's the whole problem, in one image. The data lives strictly between two flat walls, 0 and 1. The line doesn't know those walls exist. Nobody told it. We only ever taught it to minimize error, we never taught it that probabilities have limits. So it does what it's always done, follow the slope, and walk straight through the walls like they aren't there.\nWhere We Go From Here\nSo here's where we've landed. Regression is the right tool when you're trying to predict a continuous quantity. Delivery time, rent, mileage, whatever. But the moment we want the model to estimate the probability of one class or another, a plain straight line stops making sense. It has no way to respect the floor at 0 or the ceiling at 1, because nothing about it was ever built to have floors or ceilings in the first place.\nWhat we actually need is something that behaves like a line everywhere in the middle, but bends and flattens out the moment it gets close to 0 or close to 1. Something that can climb, but never quite touches the ceiling. Something that can fall, but never quite touches the floor.\nThat shape has a name, and it's one of the most useful little curves in all of machine learning. It's called the sigmoid function, and it's exactly what we're building next.\nThe full code for this post is up on GitHub here:\nhttps://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Regression_Fails_At_Classification.ipynb\nSee you in the next one.\n#\nmachine-learning\n#\nclassification\n#\nai\n#\nartificial-intelligence\n#\nsigmoid-function\n#\nregression\n#\ndata-science\n7\nviews", "metadata": { "title": "Why Linear Regression Fails for Classification", "description": null, "keywords": null, "author": null }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/9952ce2ffbf4/code/body-000.txt", "chars": 91, "language": "text" }, { "file": "attachments/9952ce2ffbf4/code/body-001.txt", "chars": 21, "language": "text" }, { "file": "attachments/9952ce2ffbf4/code/body-002.txt", "chars": 86, "language": "text" }, { "file": "attachments/9952ce2ffbf4/code/body-003.txt", "chars": 41, "language": "text" }, { "file": "attachments/9952ce2ffbf4/code/body-004.txt", "chars": 18, "language": "text" } ], "images": [ { "status": "ok", "url": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/f973bad2-cdbd-49f4-8c65-efcfda76b78b.png", "bytes": 1298258, "reused": true, "local_path": "attachments/9952ce2ffbf4/images/002.png", "src": "https://cdn.hashnode.com/uploads/covers/6a3f9a25a837e83b51cc8712/f973bad2-cdbd-49f4-8c65-efcfda76b78b.png", "alt": "From Lines to Decisions: Why Regression Fails at Classification", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://tharunpandya.hashnode.dev/from-lines-to-decisions-why-regression-fails-at-classification.md", "label": "View as Markdown", "type": "a" }, { "href": "https://hashnode.com/@tharun-pandya", "label": "T", "type": "a" }, { "href": "https://linkedin.com/in/tharun-pandya-kodi", "label": "", "type": "a" }, { "href": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Regression_Fails_At_Classification.ipynb", "label": "https://github.com/tharun-pandya/Machine-Learning-Blog-Series/blob/main/Regression_Fails_At_Classification.ipynb", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/machine-learning", "label": "# machine-learning", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/classification", "label": "# classification", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/ai", "label": "# ai", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/artificial-intelligence", "label": "# artificial-intelligence", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/sigmoid-function", "label": "# sigmoid-function", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/regression", "label": "# regression", "type": "a" }, { "href": "https://tharunpandya.hashnode.dev/tag/data-science", "label": "# data-science", "type": "a" } ], "retry_at": "2026-10-03T08:04:40", "repair_version": 2, "repaired_at": "2026-10-14T10:50:26Z", "source_url": "https://tharunpandya.hashnode.dev/from-lines-to-decisions-why-regression-fails-at-classification", "papers": [], "actual_video_files": [], "body_chars": 8552, "expected_body_images": 1, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/9952ce2ffbf4/pages/source.html", "local_html": "attachments/9952ce2ffbf4/pages/article.html", "body_base_path": "attachments/9952ce2ffbf4/pages/", "body_markdown_path": "attachments/9952ce2ffbf4/pages/article.md" }