{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 9: Text Classification with Naive Bayes and Logistic Regression",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-9-text-classification-with-naive-bayes-and-logistic-regression-2f0i",
  "publish_date": "2026-09-02",
  "corpus_stage": "",
  "notes": "",
  "id": "6dcf467b8b95",
  "collected_at": "2026-10-02T13:50:13",
  "title": "Day 9: Text Classification with Naive Bayes and Logistic Regression - DEV Community",
  "body_markdown": "Previously, on Day 8:\nExplained Named Entity Recognition (NER), detailing its role in finding and categorizing named entities in text, standard entity types, rule-based and machine learning approaches, practical code examples, and real-world applications and challenges.\nWhat is Text Classification?\nText classification is the process of automatically assigning categories or labels to text. For example, a system might learn to recognize whether an email is \"spam\" or \"not spam,\" or determine whether a tweet has \"positive,\" \"negative,\" or \"neutral\" sentiment.\nThis goes beyond tasks like tokenization (splitting text into words) or named entity recognition (identifying names and dates). Those focus on extracting specific elements. In contrast, text classification tries to capture the overall meaning or purpose of the entire text and assign a single label to it.\nText classification appears in many places. Spam filters use it to block junk email. News sites use it to sort articles by topic. Apps use it to route support tickets to the correct department. If you want to sort text into buckets based on what it means, you need text classification.\nNaive Bayes: A Simple, Classic Approach\nNaive Bayes is one of the oldest and simplest algorithms for classification. It's a good starting point for text tasks.\nThe core idea is probability. The algorithm asks: \"Given this text, what is the probability it belongs to each possible category, based on which words are present?\" For instance, in spam detection: if a message contains \"free,\" \"win,\" and \"money,\" what are the chances it's spam?\nThe \"naive\" part refers to a major simplifying assumption: it treats each feature—here, each word—as if it’s independent of the others, given the label. This means the presence of \"free\" is assumed unrelated to the presence of \"money.\" In real text, that's rarely true, but the method often still works well. Word usage patterns are strong enough that this rough assumption still separates categories effectively.\nPreparing Text Data for Naive Bayes\nMachine learning models—including Naive Bayes—work with numbers, not raw text. So, each example needs to be converted into numbers. The most common approach for Naive Bayes is called the bag-of-words model.\nBag-of-words means counting how many times each word appears in a chunk of text, ignoring word order and grammar. For example:\n\"Free money now!\" → {\"free\": 1, \"money\": 1, \"now\": 1}\nIf we're building a spam filter, we scan all available spam and non-spam texts, count word occurrences for each group, and then compute the probability of each word appearing in spam vs. non-spam.\nNote:\nIn spam detection, there are usually two labels:\n\"spam\"\n: unwanted, promotional, or fraudulent messages\n\"ham\"\n: a standard term for messages that are not spam (i.e., regular, wanted messages)\nWhen a new message arrives, Naive Bayes looks at its words, combines the probabilities (using Bayes’ Rule), and chooses the most likely label.\nNaive Bayes for Spam Detection: Minimal Example\nHere’s a minimal, real code example using scikit-learn (sklearn). We'll use some sample SMS messages, convert them to bag-of-words vectors, and train/test a Naive Bayes classifier.\nfrom\nsklearn.feature_extraction.text\nimport\nCountVectorizer\nfrom\nsklearn.naive_bayes\nimport\nMultinomialNB\nfrom\nsklearn.model_selection\nimport\ntrain_test_split\nfrom\nsklearn.metrics\nimport\naccuracy_score\n# Simple SMS data. For real tasks, use much more data!\ntexts\n=\n[\n\"\nWin a free iPhone now\n\"\n,\n\"\nCall me as soon as you can\n\"\n,\n\"\nCheap meds available online\n\"\n,\n\"\nHey, are we meeting today?\n\"\n,\n\"\nCongratulations! You won cash.\n\"\n,\n\"\nLet\n'\ns catch up for lunch.\n\"\n]\nlabels\n=\n[\n\"\nspam\n\"\n,\n\"\nham\n\"\n,\n\"\nspam\n\"\n,\n\"\nham\n\"\n,\n\"\nspam\n\"\n,\n\"\nham\n\"\n]\n# Indices to keep track of which text is which\ntext_indices\n=\nlist\n(\nrange\n(\nlen\n(\ntexts\n)))\n# Step 1: Convert text to bag-of-words counts\nvectorizer\n=\nCountVectorizer\n()\nX\n=\nvectorizer\n.\nfit_transform\n(\ntexts\n)\n# Show how a text line becomes a vector\nsample\n=\ntexts\n[\n0\n]\nprint\n(\nf\n\"\nSample text:\n'\n{\nsample\n}\n'\"\n)\nsample_vec\n=\nvectorizer\n.\ntransform\n([\nsample\n]).\ntoarray\n()\nprint\n(\n\"\nVector representation:\n\"\n,\nsample_vec\n)\nprint\n(\n\"\nVocabulary mapping:\n\"\n,\nvectorizer\n.\nvocabulary_\n)\n# Step 2: Train/test split (keep indices to trace back to text)\nX_train\n,\nX_test\n,\ny_train\n,\ny_test\n,\nidx_train\n,\nidx_test\n=\ntrain_test_split\n(\nX\n,\nlabels\n,\ntext_indices\n,\ntest_size\n=\n0.5\n,\nrandom_state\n=\n42\n)\n# Step 3: Train Naive Bayes model\nnb_clf\n=\nMultinomialNB\n()\nnb_clf\n.\nfit\n(\nX_train\n,\ny_train\n)\n# Step 4: Make predictions and evaluate\ny_pred\n=\nnb_clf\n.\npredict\n(\nX_test\n)\nprint\n(\n\"\nNaive Bayes accuracy:\n\"\n,\naccuracy_score\n(\ny_test\n,\ny_pred\n))\n# Show predictions for each text\nfor\ni\nin\nrange\n(\nlen\n(\nX_test\n)):\ntext\n=\ntexts\n[\nidx_test\n[\ni\n]]\nlabel\n=\ny_test\n[\ni\n]\npred\n=\ny_pred\n[\ni\n]\nprint\n(\nf\n\"\nMSG:\n'\n{\ntext\n}\n'\n| TRUE:\n{\nlabel\n}\n| NB-PREDICTED:\n{\npred\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nHow it works:\nThe vectorizer transforms each text into a vector of word counts (the bag-of-words).\nSplitting into train and test sets checks that the model generalizes.\nTraining fits word probabilities for spam and ham.\nPrediction multiplies those probabilities using Bayes’ Rule to select the best label.\nLogistic Regression: A Different Orientation\nLogistic regression is another core method for classification. Despite its name, it’s used for classification (categorizing classes), not regression (predicting numbers). Naive Bayes models the probability of seeing this text if it's \"spam\"; logistic regression directly models the chance that this text\nis\nspam, given the features.\nYou can think of logistic regression as fitting a line (technically, a weighted sum plus a sigmoid function to squash the score between 0 and 1) to separate classes. Each word is given a “weight” that pushes a message toward \"spam\" or \"ham.\" More of a spammy word increases the spam probability; less of it decreases it.\nWhy use logistic regression? It often achieves higher accuracy than Naive Bayes when you have a decent amount of data, because it can model subtle, overlapping patterns. Its output can be directly interpreted as predicted probabilities.\nLogistic Regression Example\nLet’s run nearly the same workflow as above, but swap in logistic regression. (You’ll need to have run the previous code block so all variables exist.)\nfrom\nsklearn.linear_model\nimport\nLogisticRegression\nlr_clf\n=\nLogisticRegression\n(\nmax_iter\n=\n1000\n)\nlr_clf\n.\nfit\n(\nX_train\n,\ny_train\n)\ny_pred_lr\n=\nlr_clf\n.\npredict\n(\nX_test\n)\nprint\n(\n\"\nLogistic Regression accuracy:\n\"\n,\naccuracy_score\n(\ny_test\n,\ny_pred_lr\n))\n# Compare predictions side by side\nfor\ni\nin\nrange\n(\nlen\n(\nX_test\n)):\ntext\n=\ntexts\n[\nidx_test\n[\ni\n]]\nlabel\n=\ny_test\n[\ni\n]\nnb_prediction\n=\ny_pred\n[\ni\n]\nlr_prediction\n=\ny_pred_lr\n[\ni\n]\nprint\n(\nf\n\"\nMSG:\n'\n{\ntext\n}\n'\n| TRUE:\n{\nlabel\n}\n| NB:\n{\nnb_prediction\n}\n| LR:\n{\nlr_prediction\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nWhat’s happening:\nText to numbers (vectorization) is identical to Naive Bayes.\nLogistic regression learns weights per word instead of word probabilities.\nThe evaluation process is the same.\nPredictions may be different when the labels are ambiguous. Logistic regression’s flexibility helps when classes overlap or the decision isn't obvious.\nComparing Naive Bayes and Logistic Regression\nBoth methods:\nUse simple numeric representations of text (like bag-of-words).\nOutput predictions for each class (with confidence).\nTrain and predict quickly on small or mid-sized data.\nKey differences:\nAssumptions\n: Naive Bayes assumes that each word is independent given the label; logistic regression does not make this assumption.\nWhat’s modeled\n: Naive Bayes estimates P(words | label); logistic regression estimates P(label | words).\nSpeed\n: Naive Bayes is typically faster and uses less memory, especially with very large vocabularies.\nInterpretability\n: Logistic regression’s weights for each word can be easier to read and explain.\nAccuracy\n: Logistic regression usually outperforms Naive Bayes when you have lots of data and complex boundaries. Naive Bayes works very well when word patterns are clear or the dataset is small.\nWhen to use each:\nNaive Bayes\nis a solid baseline—simple, sturdy, and hard to overfit. Great for quick tests and noisy or limited data.\nLogistic Regression\nis often better for subtle separations or if you add richer features (like word pairs, or TF-IDF weightings).\nLimits:\nBoth ignore word order unless you create features to handle it (such as bigrams).\nBoth struggle with subtleties like sarcasm or complex context that goes beyond words alone.\nIn practice: Start with Naive Bayes for a first pass. If you have more data or need more nuanced decisions, try logistic regression next. Both are solid foundations for more complex models you'll encounter later.\nKey Takeaways\nText classification assigns entire texts to categories like 'spam' or 'ham.'\nBag-of-words converts text into numeric vectors by counting word occurrences.\nNaive Bayes models word probabilities assuming word independence, making it fast and simple.\nLogistic regression assigns weights to words and often performs better with larger, complex datasets.\nBoth methods ignore word order and may struggle with subtle language cues.\nTry It Yourself\nDownload a small labeled dataset of SMS or email messages. Split the data into training and test sets. Train both a Naive Bayes and a logistic regression classifier to identify spam. Compare their test accuracies and review some cases where the models give different predictions, reflecting on possible reasons for disagreement.\nFurther Resources\n🎥\nText Classification Explained: Naive Bayes, Logistic Regression, & Fine‑Tuned BERT\n📘\nscikit‑learn documentation: Naive Bayes (MultinomialNB, BernoulliNB, ComplementNB)\n📄\nLecture 10 – Text Classification with Bag‑of‑Words and TF‑IDF in NLP\n📄\nText Classification from Scratch: TF‑IDF and Naive Bayes\nComing up on Day 10:\nEvaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix",
  "metadata": {
    "title": "Day 9: Text Classification with Naive Bayes and Logistic Regression - DEV Community",
    "description": "Previously, on Day 8: Explained Named Entity Recognition (NER), detailing its role in finding and... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-9-text-classification-with-naive-bayes-and-logistic-regression-2f0i",
    "og:title": "Day 9: Text Classification with Naive Bayes and Logistic Regression",
    "og:description": "Previously, on Day 8: Explained Named Entity Recognition (NER), detailing its role in finding and...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fow7t082qje6cjx3uqt6d.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 9: Text Classification with Naive Bayes and Logistic Regression",
    "twitter:description": "Previously, on Day 8: Explained Named Entity Recognition (NER), detailing its role in finding and...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fow7t082qje6cjx3uqt6d.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/6dcf467b8b95/code/body-000.txt",
      "chars": 1608,
      "language": "text"
    },
    {
      "file": "attachments/6dcf467b8b95/code/body-001.txt",
      "chars": 503,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=1jsTRXZTtq8",
      "src": "https://www.youtube.com/watch?v=1jsTRXZTtq8",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/6dcf467b8b95/videos/repair-1jsTRXZTtq8.mp4",
      "bytes": 25298330,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "636.901000",
      "reused": false
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=1jsTRXZTtq8",
      "label": "Text Classification Explained: Naive Bayes, Logistic Regression, & Fine‑Tuned BERT",
      "type": "a"
    },
    {
      "href": "https://scikit-learn.org/stable/modules/naive_bayes.html",
      "label": "scikit‑learn documentation: Naive Bayes (MultinomialNB, BernoulliNB, ComplementNB)",
      "type": "a"
    },
    {
      "href": "https://electuresai.com/lecture-10-text-classification-bag-of-words-tf-idf/",
      "label": "Lecture 10 – Text Classification with Bag‑of‑Words and TF‑IDF in NLP",
      "type": "a"
    },
    {
      "href": "https://sesen.ai/blog/text-classification-tfidf-naive-bayes",
      "label": "Text Classification from Scratch: TF‑IDF and Naive Bayes",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:11Z",
  "source_url": "https://dev.to/priyeshdave6/day-9-text-classification-with-naive-bayes-and-logistic-regression-2f0i",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 10020,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/6dcf467b8b95/pages/source.html",
  "local_html": "attachments/6dcf467b8b95/pages/article.html",
  "body_base_path": "attachments/6dcf467b8b95/pages/",
  "body_markdown_path": "attachments/6dcf467b8b95/pages/article.md"
}