{ "source_file": "0930", "source_platform": "Dev.to", "author": "amananandrai", "title_listed": "Natural Language Processing 101", "keywords": "", "domain": "文本处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://dev.to/amananandrai/natural-language-processing-101-1aeo", "publish_date": "2021-02-04", "corpus_stage": "", "notes": "", "id": "8da586f25819", "collected_at": "2026-10-02T14:02:28", "title": "Natural Language Processing 101 - DEV Community", "body_markdown": "Introduction\nHumans have tried to communicate with computers since the advent of these smart devices. Initially, punch cards were used to talk to large devices like ENIAC and UNIVAC, and then programming languages were developed for communicating with computer systems. The ultimate goal is to talk to humans or instruct them in the human-understandable language which is known as\nNatural Language\n. At the start of human-computer interaction, the languages were Assembly languages and then COBOL and FORTRAN which were easily interpreted by the machines but were difficult to understand by humans. Then programming languages were developed which were more human-understandable like SQL and Python. The number of natural languages present in the world is very large but the main focus of the research community is related to English.\nIn the present time, we can give voice commands to the computer systems and they respond to them. This is what we as humans are trying to achieve. In the commercial of a famous Automobile brand when the car understands the commands of its owner it is termed as \"A Human Thing\". We humans, try to talk to people to grow a bond with them. We also talk with our pets and even in some cases to non-living objects as well. If we receive a response from these things we feel a sense of connection with them. Communication helps in bonding. And if it is in the natural language then it is an immense joy.\nNatural Language Processing\nNatural Language Processing\nis the subset of Machine Learning which deals with the interpretation of natural language described above. NLP the abbreviation for Natural Language Processing is very famous among the Data Science and Machine Learning community. NLP helps to bypass programming languages to give commands to systems and allows us to use our voice and speech to give instructions. It basically breaks down the barriers of communication by allowing anyone, whether they have computing knowledge or not, to talk to bots, systems, apps, or any kind of software. The field of NLP is vast and research is constricted to just English but there are many languages present in the world and if we are able to make computers understand our local languages like Spanish, French, German, Afrikaans, Hindi, Tamil, Bengali, etc, then it would make the world a much better place. Computers would be able to assist humans in their daily tasks and even make the life of ordinary humans quite easy. It would not be necessary for common people to instruct computers to perform certain tasks to know about Programming languages. The general person can just say the commands and the computer performs the tasks.\nNLP and its relation to Machine Learning\nMachine learning, by definition, is a type of artificial intelligence that provides computers the ability to learn without being explicitly programmed. Machine learning finds patterns in data and based on that provides results. Machine learning can help NLP powered systems adjust actions according to the historical context and patterns it picks up in a conversation. Thus ML is one of the most important parts of NLP powered systems. The increase in the amount of data helps NLP models to better emulate human languages.\nThings to learn for starting with NLP\nThere are different aspects of NLP. It has basically two parts -\nNatural language generation (NLG)\nand\nNatural language generation\n. The name itself explains that NLG means to generate sentences(language) and NLU means to understand the natural language of humans. GPT-3 which was one of the most famous word a few months ago is an NLG model which generates sentences based on given input and BERT which is a model used in Google search engine for NLU to process our queries and give answers based on them.\nSome of the things whose knowledge can help in our road to glory in the path of NLP learning are-\nBasic Linguistics\nString Manipulation\nRegular expressions\nData cleaning\nText analysis\nMachine learning and Deep learning basics\nBasic Linguistics\nis one of the essential parts of NLP apart from technical knowledge. One must know the basics of the language in which they are comfortable to build models for NLP. Knowledge of\nString Manipulation\nand\nRegular Expressions\nis required for starting with NLP. We are dealing with sentences and have to find patterns so string manipulation and regular expressions are required.\nData Cleaning\nis one of the most important parts of any data science field. In NLP also it has a major role. We remove stopwords, emojis, punctuation marks to make Bag-of-Words which are used to train models. These things are done by the use of string manipulation and the help of regular expressions.\nThe most important part is\nText Analysis\nin the NLP business. Various steps are there in the NLP pipeline which constitutes the Text analysis phase. Some of them are making\nn-grams\n. It is a collection of n consecutive words in a sentence which are used for finding the probability of the next word in sentence completion tasks or to fill up the missing words in sentences which are masked.\nTokenization\nis another step in which the sentences are divided into tokens which consists of words, punctuation marks, etc.\nStemming\nand\nLemmatization\nare another steps in NLP pipeline, in Stemming we crop the words like 'cars' to 'car' and 'walking','walked' to 'walk' whereas in lemmatization means to get the root word using grammatical laws. An example of lemmatization is converting 'is', 'am', 'are' to 'be' as it is the root of all these words. One more important step is\nPart of Speech(POS) tagging\nin which the parts of speech of the different parts of sentences is identified. After performing these steps Machine learning and Deep Learning algorithms are used to perform various NLP tasks like Sentiment Analysis, Text Summarisation, Machine Translation, Question Answering, etc.\nI hope this article has aroused interest in beginners about NLP and some more people will try to learn about NLP in near future. Some more articles related to NLP written by me are-\nPython NLP libraries to learn and use in 2021\namananandrai ・ Jan 9 '21\n#machinelearning\n#nlp\n#python\n5 NLP tasks using Hugging Face pipeline\namananandrai ・ Oct 22 '20\n#nlp\n#machinelearning\n#python\n#tutorial\nLanguage Model Implementation (Bigram Model)\namananandrai ・ Aug 8 '20\n#machinelearning\n#nlp\n#python\n#tutorial\nTimeline of advances in the field of NLP that led to development of tools like ChatGPT\namananandrai ・ Jun 4 '20\n#nlp\n#machinelearning\n#ai", "metadata": { "title": "Natural Language Processing 101 - DEV Community", "description": "Introduction Humans have tried to communicate with computers since the advent of these sma... Tagged with nlp, machinelearning, beginners.", "keywords": "nlp, machinelearning, beginners, software, coding, development, engineering, inclusive, community", "author": null, "og:type": "article", "og:url": "https://dev.to/amananandrai/natural-language-processing-101-1aeo", "og:title": "Natural Language Processing 101", "og:description": "Introduction Humans have tried to communicate with computers since the advent of these sma...", "og:site_name": "DEV Community", "og:image": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fi%2Fjtppg4nq73flfblvhizu.png", "twitter:site": "@thepracticaldev", "twitter:creator": "@", "twitter:title": "Natural Language Processing 101", "twitter:description": "Introduction Humans have tried to communicate with computers since the advent of these sma...", "twitter:card": "summary_large_image", "twitter:widgets:new-embed-design": "on", "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fi%2Fjtppg4nq73flfblvhizu.png" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [], "images": [ { "status": "ok", "url": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F355422%2Fd9bae3e1-e185-4e93-a850-58be1b042695.png", "bytes": 63526, "reused": true, "local_path": "attachments/8da586f25819/images/003.png", "src": "https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F355422%2Fd9bae3e1-e185-4e93-a850-58be1b042695.png", "alt": "amananandrai", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://dev.to/amananandrai", "label": "", "type": "a" }, { "href": "https://dev.to/amananandrai/python-nlp-libraries-to-learn-and-use-in-2021-2odp", "label": "Python NLP libraries to learn and use in 2021 amananandrai ・ Jan 9 '21 #machinelearning #nlp #python", "type": "a" }, { "href": "https://dev.to/amananandrai/5-nlp-tasks-using-hugging-face-pipeline-5b98", "label": "5 NLP tasks using Hugging Face pipeline amananandrai ・ Oct 22 '20 #nlp #machinelearning #python #tutorial", "type": "a" }, { "href": "https://dev.to/amananandrai/language-model-implementation-bigram-model-22ij", "label": "Language Model Implementation (Bigram Model) amananandrai ・ Aug 8 '20 #machinelearning #nlp #python #tutorial", "type": "a" }, { "href": "https://dev.to/amananandrai/recent-advances-in-the-field-of-nlp-33o1", "label": "Timeline of advances in the field of NLP that led to development of tools like ChatGPT amananandrai ・ Jun 4 '20 #nlp #ma", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:49:20Z", "source_url": "https://dev.to/amananandrai/natural-language-processing-101-1aeo", "papers": [], "actual_video_files": [], "body_chars": 6510, "expected_body_images": 1, "discovery_status": "ok", "delivery_status": "ok", "missing_reason": "", "raw_html": "attachments/8da586f25819/pages/source.html", "local_html": "attachments/8da586f25819/pages/article.html", "body_base_path": "attachments/8da586f25819/pages/", "body_markdown_path": "attachments/8da586f25819/pages/article.md" }