{
  "source_file": "0930",
  "source_platform": "研究机构",
  "author": "Anthropic Research",
  "title_listed": "Constitutional AI: Harmlessness from AI Feedback",
  "keywords": "宪法AI,对齐,安全",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback",
  "publish_date": "2022-12",
  "corpus_stage": "强化学习/对齐",
  "notes": "宪法AI：从AI反馈学习无害性",
  "id": "b0b4d5de1fde",
  "collected_at": "2026-10-02T15:28:55",
  "title": "Constitutional AI: Harmlessness from AI feedback \\ Anthropic",
  "body_markdown": "Alignment\nResearch\nConstitutional AI: Harmlessness from AI feedback\nDec 15, 2022\nRead Paper\nAbstract\nAs AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.\nPolicy Memo\nConstitutional AI Policy Memo\nRelated content\nAI models may be capable of self-replicating without access to a datacenter\nOur experiments suggest that small AI models may be only months away from being capable enough to self-replicate on commercial IoT devices, like Android TVs and smart ovens.\nRead more\nStrengthening our defenses against distillation attacks\nIllicit distillation allows model developers to copy frontier AI capabilities without creating proportionally robust safeguards. Here, we describe how we’ve reduced the rate of distillation by more than 90%.\nRead more\nInvestigating unintended model actions in our evaluations and internal use\nThis report describes examples of unintended model actions we’ve observed during evaluations and internal use of Claude.\nRead more",
  "metadata": {
    "title": "Constitutional AI: Harmlessness from AI feedback \\ Anthropic",
    "description": "Constitutional AI trains a harmless but non-evasive assistant through self-critique and RL from AI feedback, with far fewer human labels.",
    "keywords": null,
    "author": null,
    "og:title": "Constitutional AI: Harmlessness from AI feedback",
    "og:description": "Constitutional AI trains a harmless but non-evasive assistant through self-critique and RL from AI feedback, with far fewer human labels.",
    "og:image": "https://www-cdn.anthropic.com/images/4zrzovbb/website/6d4a0d28992ade92d6fa63646fd9c9d318245c6c-2400x1260.jpg",
    "og:image:alt": "Anthropic logo",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:site": "@AnthropicAI",
    "twitter:creator": "@AnthropicAI",
    "twitter:title": "Constitutional AI: Harmlessness from AI feedback",
    "twitter:description": "Constitutional AI trains a harmless but non-evasive assistant through self-critique and RL from AI feedback, with far fewer human labels.",
    "twitter:image": "https://www-cdn.anthropic.com/images/4zrzovbb/website/6d4a0d28992ade92d6fa63646fd9c9d318245c6c-2400x1260.jpg",
    "twitter:image:alt": "Anthropic logo",
    "article:published_time": "2022-12-15T08:00:00.000Z",
    "article:modified_time": "2026-09-09T19:24:37.000Z"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://arxiv.org/abs/2212.08073",
      "label": "Read Paper",
      "type": "a"
    },
    {
      "href": "https://www-cdn.anthropic.com/7512771452629584566b6303311496c262da1006/Anthropic_ConstitutionalAI_v2.pdf",
      "label": "Constitutional AI Policy Memo",
      "type": "a"
    },
    {
      "href": "https://twitter.com/intent/tweet?text=https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.linkedin.com/shareArticle?mini=true&url=https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/research/self-replicating-ai",
      "label": "Read more",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/research/distillation-defenses",
      "label": "Read more",
      "type": "a"
    },
    {
      "href": "https://www.anthropic.com/research/investigating-unintended-model-actions",
      "label": "Read more",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:53:35Z",
  "source_url": "https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2212.08073",
      "bytes": 2088111,
      "reused": true,
      "local_path": "papers/repair/b3e647be3bc180f47791.pdf"
    },
    {
      "status": "ok",
      "url": "https://www-cdn.anthropic.com/7512771452629584566b6303311496c262da1006/Anthropic_ConstitutionalAI_v2.pdf",
      "bytes": 207383,
      "reused": false,
      "local_path": "papers/repair/9272ee66451cf14c77fb.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 2208,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/b0b4d5de1fde/pages/source.html",
  "local_html": "attachments/b0b4d5de1fde/pages/article.html",
  "body_base_path": "attachments/b0b4d5de1fde/pages/",
  "body_markdown_path": "attachments/b0b4d5de1fde/pages/article.md"
}