{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Bala Priya C",
  "title_listed": "Feature Engineering for NLP",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/balapriya/feature-engineering-for-nlp-4e1l",
  "publish_date": "2021-05-10",
  "corpus_stage": "",
  "notes": "",
  "id": "a3fdcc2edf1a",
  "collected_at": "2026-10-02T14:01:33",
  "title": "Feature Engineering for NLP - DEV Community",
  "body_markdown": "The first post in the series covered the introductory aspects of NLP, techniques for text pre-processing. This part covers linguistic aspects such as Syntax, Semantics, POS tagging, Named Entity Recognition(NER) and N-grams for language modeling.\nOutline\n-\nUnderstanding\nSyntax\n&\nSemantics\n-\nTechniques\nto\nunderstand\ntext\n--\nPOS\ntagging\n--\nUnderstanding\nEntity\nParsing\n--\nNamed\nEntity\nRecognition\n(\nNER\n)\n--\nUnderstanding\nN\n-\ngrams\nEnter fullscreen mode\nExit fullscreen mode\nAs we know, one of the key challenges in NLP is the inherent complexity in processing natural language; understanding the grammar and context(syntax and semantics), resolving ambiguity(disambiguation), co-reference resolution, etc.\nUnderstanding Syntax and Semantics\nSyntactic and Semantic Analysis are fundamental techniques to understand any natural language.\nSyntax refers to the set of rules specific to the language's grammatical structure, while Semantics refers to the meaning conveyed.\nTherefore, semantic analysis refers to the process of understanding the meaning and interpretation of words and sentence structure.\nA sentence that is syntactically correct need not always be semantically correct!\nLet us consider a funny example, where a person responds to\n“Call me a cab!”\nwith\n“OK, you’re a cab!”\nclearly misinterpreting the context, which is not meaningful. This example shows how a syntactically correct sentence (“OK, you’re a cab!” is grammatically perfect!🙂) fails to make sense.\nTechniques to understand text\nPOS Tagging\nPOS Tagging, also called\ngrammatical-tagging\nor\nword category disambiguation\n, refers to the process of marking a word in the corpus as corresponding to a particular part of speech, based on both its definition and context. The POS tags from the Penn Treebank project, which are widely used in practice, can be found in this link.\nWhy is POS Tagging important?\nPOS tagging is important for word sense disambiguation. POS tagging is of importance in applications such as machine translation and information retrieval.\nChallenges in POS tagging: Due to the inherent ambiguity in language, POS tags are not generic. “The same word may take different tags in different sentences depending on different contexts.”\nExamples\n:\nShe\nsaw\na\nbear\n(\nbear\n-\nNoun\n);\nYour\nefforts\nwill\nbear\nfruits\n(\nbear\n-\nVerb\n)\nWhere\nis\nthe\ntrash\ncan\n?\n(\ncan\n-\nNoun\n)\n;\nI\ncan\ndo\nbetter\n!\n(\ncan\n-\nModal\nverb\n)\nEnter fullscreen mode\nExit fullscreen mode\nThere are several approaches to POS tagging, such as Rule-based approaches, Probabilistic (Stochastic) POS tagging using Hidden Markov Models.\nShallow Parsing/Chunking\nShallow parsing or Chunking is the process of dividing a text into syntactically related groups. It involves dividing the text into a non-overlapping contiguous subset of tokens. Shallow parsing segments and labels a multi-token sequence. It’s important in information extraction from text to create meaningful subcomponents.\nNamed Entity Recognition(NER)\nThe goal of Named Entity Recognition(NER) is the process of automatically finding names of people, places, and organizations in text across many languages. NER is used in information extraction to identify and extract named entities in predefined classes. Named Entities in a text are those entities that are often more informative and contextually relevant.\nThe key steps involved include\nIdentifying the named entity\nExtracting the named entity\nCategorizing the named entity into tags such as PERSON, ORGANIZATION, LOCATION, etc.\nUnderstanding N-Grams\nN-Grams is a useful language model aimed at finding probability distributions over word sequences. N-Gram essentially means a sequence of N words. Consider a simple example sentence, “This is Big Data AI Book,” whose unigrams, bigrams, and trigrams are shown below.\nUnderstanding the Math\nP\n(\nw\n|\nh\n):\nProbability\nof\nword\nw\n,\ngiven\nsome\nhistory\nh\nExample\n:\nP\n(\nbecause\n|\ntoday\nI\nam\nhappy\n)\nw\n:\nbecause\nh\n:\ntoday\nI\nam\nhappy\n# Approach 1: Relative Frequency Count\nStep\n1\n:\nTake\na\ntext\ncorpus\nStep\n2\n:\nCount\nthe\nnumber\nof\ntimes\n'\ntoday I am happy\n'\nappears\nStep\n3\n:\nCount\nthe\nnumber\nof\ntimes\nit\nis\nfollowed\nby\n'\nbecause\n'\nP\n(\nbecause\n|\ntoday\nI\nam\nhappy\n)\n=\nCount\n(\ntoday\nI\nam\nhappy\nbecause\n)\n/\nCount\n(\ntoday\nI\nam\nhappy\n)\n# In essence, we seek to answer the question,\nOut\nof\nthe\nN\ntimes\nwe\nsaw\nthe\nhistory\nh\n,\nhow\nmany\ntimes\ndid\nthe\nword\nw\nfollow\nit\n?\n# Disadvantages of the approach:\nWhen\nthe\nsize\nof\nthe\ntext\ncorpus\nis\nlarge\n,\nthen\nthis\napproach\nhas\nto\ntraverse\nthe\nentire\ncorpus\n.\nNot\nscalable\nand\nis\nclearly\nsuboptimal\nin\nperformance\n.\n# Approach 2: Bigram Model\nBigram\nmodel\napproximates\nthe\nprobability\nof\na\nword\ngiven\nall\nthe\nprevious\nwords\nby\nusing\nonly\nthe\nconditional\nprobability\nof\nthe\npreceding\nword\n.\nEnter fullscreen mode\nExit fullscreen mode\nThe assumption that the probability of occurrence of a word depends only on the preceding word (Markov Assumption) is quite strong; In general, an N-grams model assumes dependence on the preceding (N-1) words. In practice, this N is a hyperparameter that we can play around with to check which N optimizes model performance on the specific task, say sentiment analysis, text classification, etc.😊\nPutting it all together, we’ve covered the differences between syntactic and semantic analysis, importance of POS tagging, Named Entity Recognition(NER) and chunking in text analysis and briefly looked at the concept of N-grams for language modeling.\nReferences\nBelow is the link to the Google Colab notebook that explains the implementation of POS Tagging, Parsing, and Named Entity Recognition(NER) on the\n‘All the News’\ndataset from kaggle that contains 143,000 articles from 15 publications.\nWomenWhoCode/WWCodeDataScience\nAll the news\nUnderstanding Word N-grams and N-gram Probability in Natural Language Processing\nAn introduction to part-of-speech tagging and the Hidden Markov\nModel\nCover Image: Photo by\nEd Robertson\non\nUnsplash",
  "metadata": {
    "title": "Feature Engineering for NLP - DEV Community",
    "description": "The first post in the series covered the introductory aspects of NLP, techniques for text pre-proces... Tagged with nlp, python, beginners, machinelearning.",
    "keywords": "nlp, python, beginners, machinelearning, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/balapriya/feature-engineering-for-nlp-4e1l",
    "og:title": "Feature Engineering for NLP",
    "og:description": "The first post in the series covered the introductory aspects of NLP, techniques for text pre-proces...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fremc1hfv5yfaju57fyws.jpg",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Feature Engineering for NLP",
    "twitter:description": "The first post in the series covered the introductory aspects of NLP, techniques for text pre-proces...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fremc1hfv5yfaju57fyws.jpg"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/a3fdcc2edf1a/code/body-000.txt",
      "chars": 188,
      "language": "text"
    },
    {
      "file": "attachments/a3fdcc2edf1a/code/body-001.txt",
      "chars": 155,
      "language": "text"
    },
    {
      "file": "attachments/a3fdcc2edf1a/code/body-002.txt",
      "chars": 896,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://www.kaggle.com/snapcrack/all-the-news",
      "label": "‘All the News’",
      "type": "a"
    },
    {
      "href": "https://github.com/WomenWhoCode/WWCodeDataScience/blob/master/Intro_to_NLP/2_NLP_DeepDive1.ipynb",
      "label": "WomenWhoCode/WWCodeDataScience",
      "type": "a"
    },
    {
      "href": "https://towardsdatascience.com/understanding-word-n-grams-and-n-gram-probability-in-natural-language-processing-9d9eef0fa058",
      "label": "Understanding Word N-grams and N-gram Probability in Natural Language Processing",
      "type": "a"
    },
    {
      "href": "https://www.freecodecamp.org/news/an-introduction-to-part-of-speech-tagging-and-the-hidden-markov-model-953d45338f24/",
      "label": "An introduction to part-of-speech tagging and the Hidden Markov\nModel",
      "type": "a"
    },
    {
      "href": "https://unsplash.com/@eddrobertson?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",
      "label": "Ed Robertson",
      "type": "a"
    },
    {
      "href": "https://unsplash.com/s/photos/books?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",
      "label": "Unsplash",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:11Z",
  "source_url": "https://dev.to/balapriya/feature-engineering-for-nlp-4e1l",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 5902,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/a3fdcc2edf1a/pages/source.html",
  "local_html": "attachments/a3fdcc2edf1a/pages/article.html",
  "body_base_path": "attachments/a3fdcc2edf1a/pages/",
  "body_markdown_path": "attachments/a3fdcc2edf1a/pages/article.md"
}