{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Day 10: Evaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/day-10-evaluation-metrics-for-nlp-precision-recall-f1-confusion-matrix-15ie",
  "publish_date": "2026-09-03",
  "corpus_stage": "",
  "notes": "",
  "id": "30bf71918eab",
  "collected_at": "2026-10-02T13:50:05",
  "title": "Day 10: Evaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix - DEV Community",
  "body_markdown": "Previously, on Day 9:\nExplained the fundamentals of text classification, focusing on bag-of-words representations and comparing Naive Bayes and logistic regression classifiers for labeling text such as spam detection.\nModel evaluation means measuring how well your NLP system performs a specific task. These measurements, called metrics, offer concrete evidence of your model’s strengths and weaknesses. For engineers, metrics are not just numbers—research, engineering choices, and business decisions all depend on them.\nAccuracy: The Basic Metric\nAccuracy tells you what fraction of predictions were correct. But accuracy alone can mislead, especially when certain mistakes cost more, or when your data is imbalanced.\nTake a spam filter. If only 5% of emails are spam, a model that always predicts “not spam” scores 95% accuracy. It looks good on paper, but in reality, the system fails at catching spam.\nAccuracy is just a starting point. Most real-world problems need more nuanced metrics.\nPrecision: Correctness of Positive Predictions\nPrecision measures how many of the items your model labeled as positive were actually positive.\nIn plain terms: if your spam filter marks 10 emails as spam, and only 8 are really spam, your model made 2 mistakes—it blocked good email.\nPrecision focuses only on \"is spam\" predictions. Out of everything marked as spam, how many truly were?\nPrecision = (True Positives) / (True Positives + False Positives)\nSpam filter example:\n8 spam emails correctly blocked (true positives)\n2 valid emails wrongly blocked (false positives)\nPrecision = 8 / (8 + 2) = 0.8, or 80%.\nHigh precision means that when your model says \"spam,\" it’s probably right. Systems tuned for high precision rarely block important messages—but may let some spam through.\nRecall: Coverage of Real Positives\nRecall measures how many actual positive items your model found.\nReturning to the spam filter. Suppose there were 12 spam emails in your inbox, but your model only caught 8.\nRecall = (True Positives) / (True Positives + False Negatives)\nHere:\nTrue positives: 8 spam emails caught\nFalse negatives: 4 spam emails missed and let through\nRecall = 8 / (8 + 4) = 8 / 12 ≈ 0.67, or 67%.\nHigh recall means the model finds nearly all the spam. But if recall is too high at the cost of precision, it may block good mail as well.\nPrecision and recall often work against each other. Improving one can lower the other.\nF1 Score: The Balance Point\nWhen you want a single measure that balances precision and recall, use the F1 score. It’s the harmonic mean—if either precision or recall is low, the F1 score drops sharply.\nF1 = 2 × (Precision × Recall) / (Precision + Recall)\nWith our example:\nPrecision = 0.8\nRecall = 0.67\nF1 = 2 × (0.8 × 0.67) / (0.8 + 0.67)\nF1 = 2 × 0.536 / 1.47 ≈ 1.072 / 1.47 ≈ 0.73, or 73%.\nIf either precision or recall is very low, the F1 score will also be low.\nConfusion Matrix: The Error Breakdown\nMetrics can be abstract. The confusion matrix makes results easy to see and analyze.\nA confusion matrix is a table showing the count of true and false outcomes for each class. For binary problems (like spam vs. not spam), it looks like this:\nPredicted: Spam\nPredicted: Not Spam\nActual: Spam\nTrue Positive (TP)\nFalse Negative (FN)\nActual: Not Spam\nFalse Positive (FP)\nTrue Negative (TN)\nTrue Positive (TP):\nSpam predicted as spam.\nFalse Positive (FP):\nNot-spam predicted as spam.\nTrue Negative (TN):\nNot-spam predicted as not-spam.\nFalse Negative (FN):\nSpam predicted as not-spam.\nFrom this table, you can always calculate:\nPrecision = TP / (TP + FP)\nRecall = TP / (TP + FN)\nHere’s a bare-bones Python example:\n# Sample labels (1=spam, 0=not spam)\nactual\n=\n[\n1\n,\n0\n,\n1\n,\n1\n,\n0\n,\n0\n,\n1\n,\n0\n,\n1\n,\n0\n]\npredicted\n=\n[\n1\n,\n0\n,\n1\n,\n0\n,\n0\n,\n1\n,\n1\n,\n0\n,\n0\n,\n0\n]\n# Compute confusion matrix entries\nTP\n=\nsum\n(\n1\nfor\na\n,\np\nin\nzip\n(\nactual\n,\npredicted\n)\nif\na\n==\n1\nand\np\n==\n1\n)\nFP\n=\nsum\n(\n1\nfor\na\n,\np\nin\nzip\n(\nactual\n,\npredicted\n)\nif\na\n==\n0\nand\np\n==\n1\n)\nTN\n=\nsum\n(\n1\nfor\na\n,\np\nin\nzip\n(\nactual\n,\npredicted\n)\nif\na\n==\n0\nand\np\n==\n0\n)\nFN\n=\nsum\n(\n1\nfor\na\n,\np\nin\nzip\n(\nactual\n,\npredicted\n)\nif\na\n==\n1\nand\np\n==\n0\n)\nprint\n(\nf\n\"\nTP:\n{\nTP\n}\n, FP:\n{\nFP\n}\n, TN:\n{\nTN\n}\n, FN:\n{\nFN\n}\n\"\n)\n# Now calculate the metrics\nprecision\n=\nTP\n/\n(\nTP\n+\nFP\n)\nif\n(\nTP\n+\nFP\n)\nelse\n0\nrecall\n=\nTP\n/\n(\nTP\n+\nFN\n)\nif\n(\nTP\n+\nFN\n)\nelse\n0\nf1\n=\n2\n*\nprecision\n*\nrecall\n/\n(\nprecision\n+\nrecall\n)\nif\n(\nprecision\n+\nrecall\n)\nelse\n0\nprint\n(\nf\n\"\nPrecision:\n{\nprecision\n:\n.\n2\nf\n}\n\"\n)\nprint\n(\nf\n\"\nRecall:\n{\nrecall\n:\n.\n2\nf\n}\n\"\n)\nprint\n(\nf\n\"\nF1 Score:\n{\nf1\n:\n.\n2\nf\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nThis prints the confusion matrix numbers, then precision, recall, and F1.\nMetric Pitfalls: When Numbers Mislead\nMetrics are tools. The wrong tool can lead to the wrong decisions.\nIf your data has 1,000 emails, but only 10 are spam, always predicting \"not spam\" means 99% accuracy. But you'll miss all the spam—accuracy is useless here.\nIn medical tests, recall might matter more (catch all disease cases). In email, high precision may matter more (never lose real mail). Sometimes you want a balance (F1); sometimes you care much more about one type of error than another.\nAlways connect your metrics to your real-world goals. Analyze your confusion matrix to understand your model’s errors. Never trust a single number without asking whether it matches your needs.\nMetrics measure, compare, and drive improvement. But their value always comes from how thoughtfully you choose, compute, and interpret them.\nKey Takeaways\nAccuracy alone can be misleading, especially with imbalanced data.\nPrecision measures how often positive predictions are actually correct.\nRecall reflects how many real positives your model identifies.\nThe F1 score balances precision and recall into a single metric.\nA confusion matrix shows detailed error counts, revealing strengths and weaknesses.\nTry It Yourself\nTake these actual labels: [1, 0, 1, 1, 0, 0, 1, 0, 1, 0] and predictions: [1, 0, 1, 0, 0, 1, 1, 0, 0, 0]. Manually fill in a confusion matrix by counting true positives, true negatives, false positives, and false negatives. Then calculate precision, recall, and F1 score for this data set, showing your steps.\nFurther Resources\n🎥\nBeginner's tutorial on Precision Recall and F1 Score for Machine Learning Models | Confusion Matrix\n📄\nEvaluating Classifiers: Confusion Matrix, Precision, Recall, and F1\n📘\nModel Evaluation in Scikit‑Learn — Metrics: precision, recall, F‑beta, F1, confusion matrix\n📘\n6. Learning to Classify Text (NLTK book chapter)\nComing up on Day 11:\nWhy One-Hot Encoding Fails: The Case for Embeddings",
  "metadata": {
    "title": "Day 10: Evaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix - DEV Community",
    "description": "Previously, on Day 9: Explained the fundamentals of text classification, focusing on bag-of-words... Tagged with ai, nlp, genai, 100daysofcode.",
    "keywords": "ai, nlp, genai, 100daysofcode, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/day-10-evaluation-metrics-for-nlp-precision-recall-f1-confusion-matrix-15ie",
    "og:title": "Day 10: Evaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix",
    "og:description": "Previously, on Day 9: Explained the fundamentals of text classification, focusing on bag-of-words...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fppzdky5vi0vziva4ikwu.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Day 10: Evaluation Metrics for NLP: Precision, Recall, F1, Confusion Matrix",
    "twitter:description": "Previously, on Day 9: Explained the fundamentals of text classification, focusing on bag-of-words...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fppzdky5vi0vziva4ikwu.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/30bf71918eab/code/body-000.txt",
      "chars": 781,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=JYQupddZkzc",
      "src": "https://www.youtube.com/watch?v=JYQupddZkzc",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/30bf71918eab/videos/repair-JYQupddZkzc.mp4",
      "bytes": 98361415,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "2377.641000",
      "reused": false
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/watch?v=JYQupddZkzc",
      "label": "Beginner's tutorial on Precision Recall and F1 Score for Machine Learning Models | Confusion Matrix",
      "type": "a"
    },
    {
      "href": "https://intuitivetutorial.com/2026/08/25/evaluating-classifiers-confusion-matrix-precision-recall-and-f1/",
      "label": "Evaluating Classifiers: Confusion Matrix, Precision, Recall, and F1",
      "type": "a"
    },
    {
      "href": "https://scikit-learn.org/stable/modules/model_evaluation.html",
      "label": "Model Evaluation in Scikit‑Learn — Metrics: precision, recall, F‑beta, F1, confusion matrix",
      "type": "a"
    },
    {
      "href": "https://www.nltk.org/book/ch06.html",
      "label": "6. Learning to Classify Text (NLTK book chapter)",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:11Z",
  "source_url": "https://dev.to/priyeshdave6/day-10-evaluation-metrics-for-nlp-precision-recall-f1-confusion-matrix-15ie",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 6573,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/30bf71918eab/pages/source.html",
  "local_html": "attachments/30bf71918eab/pages/article.html",
  "body_base_path": "attachments/30bf71918eab/pages/",
  "body_markdown_path": "attachments/30bf71918eab/pages/article.md"
}