{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "amananandrai",
  "title_listed": "Facebook launches Dynaboard an evaluation-as-a-service for NLP",
  "keywords": "",
  "domain": "文本处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/amananandrai/facebook-launches-dynaboard-an-evaluation-as-a-service-for-nlp-3kah",
  "publish_date": "2021-05-24",
  "corpus_stage": "",
  "notes": "",
  "id": "62b57441eb01",
  "collected_at": "2026-10-02T14:02:22",
  "title": "Facebook launches Dynaboard an evaluation-as-a-service for NLP - DEV Community",
  "body_markdown": "In Natural Language Processing it is very difficult to gauge the performance of a model. Facebook has launched\nDynaboard\nwhich ranks state-of-the-art language models like\nBERT\n,\nRoBERTa\n,\nALBERT\n,\nT5\n, and\nDeBERTa\non four common NLP tasks. The tasks are-\nNatural Language Inference\nQuestion Answering\nSentiment Analysis\nHate Speech\nFor evaluating the models for these tasks first a new performance evaluation parameter was created that is known as\nDynascore\n.\nIt takes into consideration different metrics which include\nAccuracy\n-  how many examples did the model get right as a percentage\nCompute\n- To account for computation, we measure the number of examples that a model can process per second on its instance in our evaluation cloud\nMemory\n- We average the memory usage over the duration that the model is running, with measurements taken each N seconds\nRobustness\n- We evaluate robustness of a model's prediction by measuring changes after adding perturbations to the examples\nFairness\n- we perform perturbations of original datasets by changing, for instance, noun phrase gender (e.g., replacing “sister” with “brother”, or “he” with “they”) and by substituting names with others that are statistically predicative of another race or ethnicity. For the purposes of Dynaboard scoring, a model is considered more “fair” if its predictions don’t change after such a perturbation\nDynascore is calculated by giving different weightage to these metrics and combining them depending on the type of task. First the tasks mentioned above which form the Dynabench were solved statically. Dynaboard has helped to make this process more dynamic.\nThe objectives achieved by Dynaboard are-\nReproducibility\nAccessibility\nBackwards Compatibility\nForward Compatibility\nPrediction Costs\nTo know more about Dynaboard read the official\nFB blog\nand to know about further details of implementation read the\npaper\n.",
  "metadata": {
    "title": "Facebook launches Dynaboard an evaluation-as-a-service for NLP - DEV Community",
    "description": "In Natural Language Processing it is very difficult to gauge the performance of a model. Facebook has... Tagged with machinelearning, nlp, news.",
    "keywords": "machinelearning, nlp, news, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/amananandrai/facebook-launches-dynaboard-an-evaluation-as-a-service-for-nlp-3kah",
    "og:title": "Facebook launches Dynaboard an evaluation-as-a-service for NLP",
    "og:description": "In Natural Language Processing it is very difficult to gauge the performance of a model. Facebook has...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fckn522t4nv11um7ea18a.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Facebook launches Dynaboard an evaluation-as-a-service for NLP",
    "twitter:description": "In Natural Language Processing it is very difficult to gauge the performance of a model. Facebook has...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1000,height=500,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fckn522t4nv11um7ea18a.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://ai.facebook.com/blog/dynaboard-moving-beyond-accuracy-to-holistic-model-evaluation-in-nlp/",
      "label": "FB blog",
      "type": "a"
    },
    {
      "href": "https://dynabench.org/dynaboard.pdf",
      "label": "paper",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:49:18Z",
  "source_url": "https://dev.to/amananandrai/facebook-launches-dynaboard-an-evaluation-as-a-service-for-nlp-3kah",
  "papers": [
    {
      "status": "ok",
      "url": "https://dynabench.org/dynaboard.pdf",
      "bytes": 605863,
      "reused": false,
      "local_path": "papers/repair/f5eba1bccb0eb29c9fa3.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 1899,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/62b57441eb01/pages/source.html",
  "local_html": "attachments/62b57441eb01/pages/article.html",
  "body_base_path": "attachments/62b57441eb01/pages/",
  "body_markdown_path": "attachments/62b57441eb01/pages/article.md"
}