{ "source_file": "0930", "source_platform": "个人博客", "author": "Sebastian Ruder", "title_listed": "Why You Should Do NLP Beyond English", "keywords": "多语言,NLP,低资源", "domain": "文本处理", "content_type": "观点文章", "is_paid_flag": "否", "url": "https://www.ruder.io/nlp-beyond-english/", "publish_date": "2020-08", "corpus_stage": "数据处理", "notes": "超越英语的NLP研究价值", "id": "be98dcc0c66f", "collected_at": "2026-10-02T14:16:04", "title": "Why You Should Do NLP Beyond English", "body_markdown": "cross-lingual\nWhy You Should Do NLP Beyond English\n7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.\nSebastian Ruder\n01 Aug 2020\n•\n7 min read\nShare\nNatural language processing (NLP) research predominantly focuses on developing methods that work well for English despite the many positive benefits of working on other languages. These benefits range from an outsized societal impact to modelling a wealth of linguistic features to avoiding overfitting as well as interesting challenges for machine learning (ML).\nThere are\naround 7,000 languages\nspoken around the world. The map above (see the interactive version at\nLangscape\n) gives an overview of languages spoken around the world, with each green circle representing a native language. Most of the world's languages are\nspoken\nin\nAsia, Africa, the Pacific region and the Americas\n.\nWhile we have seen exciting progress across many tasks in natural language processing (see\nPapers with Code\nand\nNLP Progress\nfor an overview) over the last years, most such results have been achieved in English and a small set of other high-resource languages.\nIn\na previous overview\nof our\nACL 2019 tutorial on Unsupervised Cross-lingual Representation Learning\n, I've defined a resource hierarchy based on the availability of unlabelled data and labelled data online. In a recent\nACL 2020 paper\n, Joshi et al. define a taxonomy similarly based on data availability, which you can see below.\nLanguage resource distribution of\nJoshi et al. (2020).\nThe size and colour of a circle represent the number of languages and speakers respectively in each category. Colours (on the VIBGYOR spectrum;\nV\niolet–\nI\nndigo–\nB\nlue–\nG\nreen–\nY\nellow–\nO\nrange–\nR\ned) represent the total speaker population size from low (violet) to high (red).\nLanguages in categories 5 and 4 that lie at a sweet spot of having both large amounts of labelled and unlabelled data available to them are well-studied in the NLP literature. On the other hand, languages in the other groups have largely been neglected.\nIn this post, I will argue why you should work on languages other than English. Specifically, I will highlight reasons from a\nsocietal\n,\nlinguistic\n,\nmachine learning\n,\ncultural and normative\n, and\ncognitive\nperspective.\nThe societal perspective\nTechnology cannot be accessible if it is only available for English speakers with a\nstandard accent\n.\nWhat language you speak determines your access to information, education, and even human connections. Even though we think of the Internet as open to everyone, there is a\ndigital language divide\nbetween dominant languages (mostly from the Western world) and others.\nOnly a few hundred languages are represented on the web\nand speakers of minority languages are severely limited in the information available to them.\nAs many more languages are being written in informal contexts in chat apps and on social media, this divide extends to all levels of technology: At the most basic language technology level, low-resource languages lack keyboard support and spell checking (\nSoria et al., 2018\n)—and keyboard support is even rarer for languages without a widespread written tradition (\nPaterson, 2015\n). At a higher level, algorithms are biased and discriminate against speakers of\nnon-English\nlanguages\nor simply with\ndifferent\naccents\n.\nThe latter is a problem because much existing work treats a high-resource language such as English as homogeneous. Our models consequently underperform on the plethora of related linguistic subcommunities, dialects, and accents (\nBlodgett et al., 2016\n). In reality, the boundaries between language varieties are much blurrier than we make them out to be and language identification of similar languages and dialects is still a challenging problem (\nJauhiainen et al., 2018\n). For instance, even though Italian is the official language in Italy, there are around 34 regional languages and dialects spoken throughout the country.\nA continuing lack of technological inclusion will not only exacerbate the language divide but it may also drive speakers of unsupported languages and dialects to high-resource languages with better technological support, further endangering such language varieties. To ensure that non-English language speakers are not left behind and at the same time to offset the existing imbalance, to lower language and literacy barriers, we need to apply our models to non-English languages.\nThe linguistic perspective\nEven though we claim to be interested in developing general language understanding methods, our methods are generally only applied to a single language, English.\nEnglish and the small set of other high-resource languages are in many ways not representative of the world's other languages. Many resource-rich languages belong to the Indo-European language family, are spoken mostly in the Western world, and are morphologically poor, i.e. information is mostly expressed syntactically, e.g. via a fixed word order and using\nmultiple separate words\nrather than through variation at the word level.\nFor a more holistic view, we can take a look at the typological features of different languages. The\nWorld Atlas of Language Structure\ncatalogues 192 typological features, i.e. structural and semantic properties of a language. For instance, one typological feature describes the typical\norder of subject, object, and verb\nin a language. Each feature has 5.93 categories on average. 48% of all feature categories exist only in the low-resource languages of groups 0–2 above and cannot be found in languages of groups 3–5 (\nJoshi et al., 2020\n). Ignoring such a large subset of typological features means that our NLP models are potentially missing out on valuable information that can be useful for generalisation.\nWorking on languages beyond English may also help us gain new knowledge about the relationships between the languages of the world (\nArtetxe et al., 2020\n). Conversely, it can help us reveal what linguistic features our models are able to capture. Specifically, you could use your knowledge of a particular language to probe aspects that differ from English such as the use of diacritics, extensive compounding, inflection, derivation, reduplication, agglutination, fusion, etc.\nThe ML perspective\nWe encode assumptions into the architectures of our models that are based on the data we intend to apply them. Even though we intend our models to be general, many of their inductive biases are specific to English and languages similar to it.\nThe lack of any explicitly encoded information in a model does not mean that it is truly language agnostic. A classic example are n-gram language models, which perform significantly worse for languages with elaborate morphology and relatively free word order (\nBender, 2011\n).\nSimilarly, neural models often overlook the complexities of morphologically rich languages (\nTsarfaty et al., 2020\n): Subword tokenization performs poorly on languages with reduplication (\nVania and Lopez, 2017\n), byte pair encoding does not align well with morphology (\nBostrom and Durrett, 2020\n), and languages with larger vocabularies are more difficult for language models (\nMielke et al., 2019\n). Differences in grammar, word order, and syntax also cause problems for neural models (\nRavfogel et al., 2018\n;\nAhmad et al., 2019\n;\nHu et al., 2020\n). In addition, we generally assume that pre-trained embeddings readily encode all relevant information, which may not be the case for all languages (\nTsarfaty et al., 2020\n).\nThe above problems pose unique challenges for modelling structure—both on the word and the sentence level—, dealing with sparsity, few-shot learning, encoding relevant information in pre-trained representations, and transferring between related languages, among many other interesting directions. These challenges are not addressed by current methods and thus call for a new set of language-aware approaches.\nRecent models have repeatedly matched human-level performance on increasingly difficult benchmarks—that is, in English using labelled datasets with thousands and unlabelled data with millions of examples. In the process, as a community we have overfit to the characteristics and conditions of English-language data. In particular, by focusing on high-resource languages, we have prioritised methods that work well only when large amounts of labelled and unlabelled data are available.\nIn contrast, most current methods break down when applied to the data-scarce conditions that are common for most of the world's languages. Even recent advances in pre-training language models that dramatically reduce the sample complexity for downstream tasks (\nPeters et al., 2018\n;\nHoward and Ruder, 2018\n;\nDevlin et al., 2019\n;\nClark et al., 2020\n) require massive amounts of clean, unlabelled data, which is not available for most of the world's languages (\nArtetxe et al., 2020\n). Doing well with few data is thus an ideal setting to test the limitations of current models—and evaluation on low-resource languages constitutes arguably its most impactful real-world application.\nThe cultural and normative perspective\nThe data our models are trained on reveals not only the characteristics of the specific language but also sheds light on cultural norms and common sense knowledge.\nHowever, such common sense knowledge may be different for different cultures. For instance, the notion of 'free' and 'non-free' varies cross-culturally where 'free' goods are ones that anyone can use without seeking permission, such as salt in a restaurant. Taboo topics are also different in different cultures. Furthermore, cultures vary in their assessment of relative power and social distance, among many other things (\nThomas, 1983\n). In addition, many real-world situations such as ones included in the COPA dataset (\nRoemmele et al., 2011\n) do not match the direct experience of many and equally do not reflect key situations that are obvious background knowledge for many people in the world (\nPonti et al., 2020\n).\nConsequently, an agent that was only exposed to English data originating mainly in the Western world may be able to have a reasonable conversation with speakers from Western countries, but conversing with someone from a different culture may lead to pragmatic failures.\nBeyond cultural norms and common sense knowledge, the data we train a model on also reflects the values of the underlying society. As an NLP researcher or practitioner, we have to ask ourselves whether we want our NLP system to exclusively share the values of a specific country or language community.\nWhile this decision might be less important for current systems that mostly deal with simple tasks such as text classification, it will become more important as systems become more intelligent and need to deal with complex decision-making tasks.\nThe cognitive perspective\nHuman children can acquire any natural language and their language understanding ability is remarkably consistent across all kinds of languages. In order to achieve human-level language understanding, our models should be able to show the same level of consistency across languages from different language families and typologies.\nOur models should ultimately be able to learn abstractions that are not specific to the structure of any language but that can generalise to languages with different properties.\nWhat you can do\nDatasets\nIf you create a new dataset, reserve half of your annotation budget for creating the same size dataset in another language.\nEvaluation\nIf you are interested in a particular task, consider evaluating your model on the same task in a different language. For an overview of some tasks, see\nNLP Progress\nor our\nXTREME benchmark\n.\nBender Rule\nState the language you are working on\n.\nAssumptions\nBe explicit about the signals your model uses and the assumptions it makes. Consider which are specific to the language you are studying and which might be more general.\nLanguage diversity\nEstimate the language diversity of the sample of languages you are studying (\nPonti et al., 2020\n).\nResearch\nWork on methods that address challenges of low-resource languages. In the next post, I will outline interesting research directions and opportunities in multilingual NLP.\nCitation\nFor attribution in academic contexts, please cite this work as:\n@misc{ruder2020beyondenglish,\n author = {Ruder, Sebastian},\n title = {{Why You Should Do NLP Beyond English}},\n year = {2020},\n howpublished = {\\url{http://ruder.io/nlp-beyond-english}},\n}\nThanks to Aida Nematzadeh, Laura Rimell, and Adhi Kuncoro for valuable feedback on drafts of this post.", "metadata": { "title": "Why You Should Do NLP Beyond English", "description": "7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.", "keywords": null, "author": null, "og:site_name": "ruder.io", "og:type": "article", "og:title": "Why You Should Do NLP Beyond English", "og:description": "7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.", "og:url": "https://www.ruder.io/nlp-beyond-english/", "og:image": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/size/w1200/2020/07/langscape-1.png", "og:image:width": "1200", "og:image:height": "560", "twitter:card": "summary_large_image", "twitter:title": "Why You Should Do NLP Beyond English", "twitter:description": "7000+ languages are spoken around the world but NLP research has mostly focused on English. This post outlines why you should work on languages other than English.", "twitter:url": "https://www.ruder.io/nlp-beyond-english/", "twitter:image": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/size/w1200/2020/07/langscape-1.png", "twitter:label1": "Written by", "twitter:data1": "Sebastian Ruder", "twitter:label2": "Filed under", "twitter:data2": "cross-lingual, natural language processing", "twitter:site": "@seb_ruder", "article:published_time": "2020-08-01T00:00:00.000Z", "article:modified_time": "2020-08-03T10:50:10.000Z", "article:tag": "natural language processing" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 200, "error": "" }, "code_blocks": [ { "file": "attachments/be98dcc0c66f/code/body-000.txt", "chars": 192, "language": "text" } ], "images": [ { "status": "ok", "url": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/size/w2000/2020/07/langscape-1.png", "bytes": 1442764, "reused": true, "local_path": "attachments/be98dcc0c66f/images/001.png", "src": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/size/w2000/2020/07/langscape-1.png", "alt": "Why You Should Do NLP Beyond English", "downloaded": true }, { "status": "ok", "url": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/2020/06/language_data_distribution.png", "bytes": 462106, "reused": true, "local_path": "attachments/be98dcc0c66f/images/005.png", "src": "https://storage.ghost.io/c/bd/89/bd89d480-b8e0-47e6-8bae-dcb0d061b8d7/content/images/2020/06/language_data_distribution.png", "alt": "Language resource distribution of Joshi et al. (2020)", "downloaded": true } ], "audio": [], "videos": [], "links": [ { "href": "https://www.ruder.io/tag/cross-lingual/", "label": "cross-lingual", "type": "a" }, { "href": "https://www.ruder.io/author/sebastian/", "label": "", "type": "a" }, { "href": "https://www.ethnologue.com/?ref=ruder.io", "label": "around 7,000 languages", "type": "a" }, { "href": "http://langscape.umd.edu/map.php?ref=ruder.io", "label": "Langscape", "type": "a" }, { "href": "https://www.ethnologue.com/guides/ethnologue200?ref=ruder.io", "label": "spoken", "type": "a" }, { "href": "https://www.ethnologue.com/guides/continents-most-indigenous-languages?ref=ruder.io", "label": "Asia, Africa, the Pacific region and the Americas", "type": "a" }, { "href": "https://paperswithcode.com/area/natural-language-processing?ref=ruder.io", "label": "Papers with Code", "type": "a" }, { "href": "http://nlpprogress.com/?ref=ruder.io", "label": "NLP Progress", "type": "a" }, { "href": "https://ruder.io/unsupervised-cross-lingual-learning/?ref=ruder.io", "label": "a previous overview", "type": "a" }, { "href": "https://tinyurl.com/xlingual?ref=ruder.io", "label": "ACL 2019 tutorial on Unsupervised Cross-lingual Representation Learning", "type": "a" }, { "href": "https://arxiv.org/abs/2004.09095?ref=ruder.io", "label": "ACL 2020 paper", "type": "a" }, { "href": "https://en.wikipedia.org/wiki/Received_Pronunciation?ref=ruder.io", "label": "standard accent", "type": "a" }, { "href": "http://labs.theguardian.com/digital-language-divide/?ref=ruder.io", "label": "digital language divide", "type": "a" }, { "href": "https://w3techs.com/technologies/overview/content_language?ref=ruder.io", "label": "Only a few hundred languages are represented on the web", "type": "a" }, { "href": "https://www.aclweb.org/anthology/L18-1656/?ref=ruder.io", "label": "Soria et al., 2018", "type": "a" }, { "href": "https://www.researchgate.net/publication/290279777_Keyboard_layouts_Lessons_from_the_me'phaa_and_sochiapam_Chinantec_designs?ref=ruder.io", "label": "Paterson, 2015", "type": "a" }, { "href": "https://qz.com/1141122/google-translates-gender-bias-pairs-he-with-hardworking-and-she-with-lazy-and-other-examples/?ref=ruder.io", "label": "non-English", "type": "a" }, { "href": "https://journals.sagepub.com/doi/10.1068/a44674?ref=ruder.io", "label": "languages", "type": "a" }, { "href": "https://www.washingtonpost.com/graphics/2018/business/alexa-does-not-understand-your-accent/?ref=ruder.io", "label": "different", "type": "a" }, { "href": "https://www.wired.com/2017/03/voice-is-the-next-big-platform-unless-you-have-an-accent/?ref=ruder.io", "label": "accents", "type": "a" }, { "href": "https://www.aclweb.org/anthology/D16-1120/?ref=ruder.io", "label": "Blodgett et al., 2016", "type": "a" }, { "href": "https://arxiv.org/abs/1804.08186?ref=ruder.io", "label": "Jauhiainen et al., 2018", "type": "a" }, { "href": "https://en.wikipedia.org/wiki/Periphrasis?ref=ruder.io", "label": "multiple separate words", "type": "a" }, { "href": "https://wals.info/?ref=ruder.io", "label": "World Atlas of Language Structure", "type": "a" }, { "href": "https://wals.info/feature/81A?ref=ruder.io", "label": "order of subject, object, and verb", "type": "a" }, { "href": "https://www.aclweb.org/anthology/2020.acl-main.658/?ref=ruder.io", "label": "Artetxe et al., 2020", "type": "a" }, { "href": "https://journals.linguisticsociety.org/elanguage/lilt/article/view/2624.html?ref=ruder.io", "label": "Bender, 2011", "type": "a" }, { "href": "https://www.aclweb.org/anthology/2020.acl-main.660.pdf?ref=ruder.io", "label": "Tsarfaty et al., 2020", "type": "a" }, { "href": "https://arxiv.org/abs/1704.08352?ref=ruder.io", "label": "Vania and Lopez, 2017", "type": "a" }, { "href": "https://arxiv.org/abs/2004.03720?ref=ruder.io", "label": "Bostrom and Durrett, 2020", "type": "a" }, { "href": "https://www.aclweb.org/anthology/P19-1491/?ref=ruder.io", "label": "Mielke et al., 2019", "type": "a" }, { "href": "https://www.aclweb.org/anthology/W18-5412/?ref=ruder.io", "label": "Ravfogel et al., 2018", "type": "a" }, { "href": "https://www.aclweb.org/anthology/N19-1253.pdf?ref=ruder.io", "label": "Ahmad et al., 2019", "type": "a" }, { "href": "https://arxiv.org/abs/2003.11080?ref=ruder.io", "label": "Hu et al., 2020", "type": "a" }, { "href": "https://www.aclweb.org/anthology/N18-1202/?ref=ruder.io", "label": "Peters et al., 2018", "type": "a" }, { "href": "https://www.aclweb.org/anthology/P18-1031.pdf?ref=ruder.io", "label": "Howard and Ruder, 2018", "type": "a" }, { "href": "https://www.aclweb.org/anthology/N19-1423/?ref=ruder.io", "label": "Devlin et al., 2019", "type": "a" }, { "href": "https://openreview.net/forum?id=r1xMH1BtvB&ref=ruder.io", "label": "Clark et al., 2020", "type": "a" }, { "href": "https://academic.oup.com/applij/article-abstract/4/2/91/167524?redirectedFrom=fulltext&ref=ruder.io", "label": "Thomas, 1983", "type": "a" }, { "href": "https://ict.usc.edu/pubs/Choice%20of%20Plausible%20Alternatives-%20An%20Evaluation%20of%20Commonsense%20Causal%20Reasoning.pdf?ref=ruder.io", "label": "Roemmele et al., 2011", "type": "a" }, { "href": "https://arxiv.org/abs/2005.00333?ref=ruder.io", "label": "Ponti et al., 2020", "type": "a" }, { "href": "https://sites.research.google/xtreme?ref=ruder.io", "label": "XTREME benchmark", "type": "a" }, { "href": "https://thegradient.pub/the-benderrule-on-naming-the-languages-we-study-and-why-it-matters/?ref=ruder.io", "label": "State the language you are working on", "type": "a" } ], "repair_version": 2, "repaired_at": "2026-10-14T10:51:59Z", "source_url": "https://www.ruder.io/nlp-beyond-english/", "papers": [ { "status": "ok", "url": "https://arxiv.org/pdf/2004.09095", "bytes": 2874015, "reused": false, "local_path": "papers/repair/a230868ec33398ecd94f.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/1804.08186", "bytes": 1308575, "reused": false, "local_path": "papers/repair/2d756a241b935ee99597.pdf" }, { "status": "ok", "url": "https://www.aclweb.org/anthology/2020.acl-main.660.pdf?ref=ruder.io", "bytes": 273411, "reused": false, "local_path": "papers/repair/7b4df7a876cf1c0cc287.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/1704.08352", "bytes": 371374, "reused": false, "local_path": "papers/repair/218ad7359708eaf30fee.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2004.03720", "bytes": 352033, "reused": false, "local_path": "papers/repair/2d6d019daa42d22dbff1.pdf" }, { "status": "ok", "url": "https://www.aclweb.org/anthology/N19-1253.pdf?ref=ruder.io", "bytes": 500350, "reused": false, "local_path": "papers/repair/0f0d266c2a0a4b8a8957.pdf" }, { "status": "ok", "url": "https://arxiv.org/pdf/2003.11080", "bytes": 915662, "reused": false, "local_path": "papers/repair/38b7f5f2a070f3d837f3.pdf" }, { "status": "ok", "url": "https://www.aclweb.org/anthology/P18-1031.pdf?ref=ruder.io", "bytes": 995744, "reused": false, "local_path": "papers/repair/993353bb50ec4775aefb.pdf" }, { "status": "http_502", "url": "https://ict.usc.edu/pubs/Choice%20of%20Plausible%20Alternatives-%20An%20Evaluation%20of%20Commonsense%20Causal%20Reasoning.pdf?ref=ruder.io", "retried_at": "2026-10-20T22:24:39Z" }, { "status": "ok", "url": "https://arxiv.org/pdf/2005.00333", "bytes": 627776, "reused": false, "local_path": "papers/repair/7e6a8e1028aa37d92404.pdf" } ], "actual_video_files": [], "body_chars": 12716, "expected_body_images": 2, "discovery_status": "ok", "delivery_status": "partial", "missing_reason": "http_502", "raw_html": "attachments/be98dcc0c66f/pages/source.html", "local_html": "attachments/be98dcc0c66f/pages/article.html", "body_base_path": "attachments/be98dcc0c66f/pages/", "body_markdown_path": "attachments/be98dcc0c66f/pages/article.md" }