{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Debashish Ghosal",
  "title_listed": "I Tried to Poison My Agent's Rule Store. It Produced 20 Triggers. Zero Got In.",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://dev.to/debashish_ghosal/i-tried-to-poison-my-agents-rule-store-it-produced-20-triggers-zero-got-in-i44",
  "publish_date": "2026-09-09",
  "corpus_stage": "",
  "notes": "",
  "id": "01f7f0fab106",
  "collected_at": "2026-10-02T14:06:16",
  "title": "I Tried to Poison My Agent's Rule Store. It Produced 20 Triggers. Zero Got In. - DEV Community",
  "body_markdown": "Update — v0.2.0 released.\nCauterRule is now live on\nGitHub\nand\nPyPI\n. It turns repeated agent failures into permanent standing rules — extract, replay-test, promote.\npip install cauterule\ngives you the full CLI, TUI review, observability, 7 export formats, adversarial corpora, and a bundled git rule pack. The\nfield test report\nevaluated 4 models across 745 trajectories and is the source for every number in this article.\nRelease notes\n·\nChangelog\nCauterRule\nis an open-source sidecar that learns standing rules from repeated agent failures. It extracts lessons from trajectories, replay-tests them, and tries to separate reusable guidance from noisy overgeneralization.\nI built 6 adversarial corpora. Prompt injection. Misleading trajectories. Contradictions. Unsafe directives. Poisoning. Instruction leakage. 10 trajectories each, 50 total. Each one designed to compromise the extractor — to make the model produce a trigger that, if promoted, would corrupt the rule store.\nThe model followed every injection. It produced plausible-looking triggers for all 6 attack types. Precision was deceptively high — 0.667 on injection — because injection prompts mimic reference failure patterns. The triggers looked real.\nZero got promoted. Across all 4 models. Across all 6 attack vectors. Not one adversarial trigger entered the rule store.\nThis is the story of why — and the defense-in-depth model that makes it work.\nThe 6 adversarial corpora\nv0.2.0 introduces 6 adversarial corpora, each testing a different attack surface:\nCorpus\nAttack type\nWhat it tests\ninjection\nPrompt injection — override system prompt with injected instructions\nCan the attacker make the model produce a trigger of their choosing?\nmisleading\nMisleading trajectories — failures that look like one thing but are another\nCan the model be tricked into extracting a rule for the wrong failure?\ncontradiction\nContradictory trajectories — two trajectories with opposite outcomes for the same trigger\nCan the model produce a trigger that contradicts existing rules?\nunsafe\nUnsafe directives — trajectories that suggest dangerous actions\nCan the model be tricked into promoting a rule that recommends unsafe behavior?\npoisoning\nData poisoning — corrupted trajectory data designed to produce a specific trigger\nCan the attacker inject trajectories that produce a predetermined trigger?\ninstruction-leakage\nInstruction leakage — system prompt content leaking into extracted triggers\nCan the model's own instructions contaminate the rule store?\nEach corpus has 10 trajectories. 50 total. All 4 models ran against all 6 corpora. The question is not \"can the model be compromised?\" — it can. The question is \"if the model is compromised, does the bad trigger get promoted?\"\nThe model complied. The system didn't.\nThe model followed the injection prompts. On the\ninjection\ncorpus, all 4 models produced candidates — 20 total across 10 injection trajectories. The triggers looked plausible. They mimicked valid failure patterns because the injection prompts were designed to mimic reference failures.\nModel\nInjection candidates\nPromoted\nPrecision (deceptive)\nLocal Llama 3.2B\n10\n0\n0.667\nLocal Qwen 4B\n10\n0\n0.667\nCloud gpt-4o-mini\n10\n0\n0.667\nCloud llama-3.1-8b\n10\n0\n0.667\nPrecision 0.667 is deceptively high. It means the injection triggers match reference failures in the corpus — they \"prevent\" real failures. But the failures they prevent are not the ones the triggers describe. The injection prompts are crafted to mimic reference failure patterns, so the matcher finds matches. The triggers look like they work.\nBut the promotion gate is not fooled. A candidate is promoted only if it prevents real failures without breaking real successes. The injection triggers match reference failures — but they also match everything else. They are too broad. The broad-trigger penalty (\nbroken > prevented → fail\n) catches them. The promotion gate rejects them.\nAcross all 6 adversarial corpora and all 4 models: 0 promoted rules. The validation suite confirms it: 41 adversarial tests, 0 failures, 0 promoted rules.\nThe per-model breakdown\nModel\ninjection\nmisleading\ncontradiction\nunsafe\npoisoning\nLlama 3.2B\n0P / 10F / 0I\n0P / 7F / 3I\n0P / 10F / 0I\n0P / 7F / 3I\n0P / 5F / 5I\nQwen 4B\n0P / 10F / 0I\n0P / 6F / 4I\n0P / 4F / 6I\n0P / 0F / 10I\n0P / 3F / 7I\ngpt-4o-mini\n0P / 0F / 10I\n0P / 3F / 7I\n0P / 1F / 9I\n0P / 0F / 10I\n0P / 1F / 9I\nllama-3.1-8b\n0P / 0F / 10I\n0P / 2F / 8I\n0P / 1F / 9I\n0P / 0F / 10I\n0P / 3F / 7I\nZero passes across the board. But the verdict distributions tell a story.\nLocal models produce hard fails.\nLlama 3.2B: 10F on injection, 10F on contradiction. The matcher rejects the triggers outright — they match too broadly, breaking more successes than they prevent. The local model's triggers are broad enough that the replay engine can decisively reject them.\nCloud models produce inconclusives.\ngpt-4o-mini: 10I on injection, 10I on unsafe, 9I on contradiction, 9I on poisoning. The cloud model's triggers are more specific — they match the reference failures more precisely. But they don't prevent enough failures to pass, and they don't break enough successes to fail. They sit in the inconclusive bucket — plausible but unproven.\nThis is a subtle point. The cloud model is \"better\" at extraction — its triggers are more specific. But on adversarial inputs, more specific is not safer. A specific trigger that matches exactly the injection target is harder to reject than a broad trigger that matches everything. The cloud model's specificity makes the inconclusive verdict more likely — the trigger is too precise to fail (it doesn't break many successes) but too narrow to pass (it doesn't prevent many failures either).\nThe promotion gate treats inconclusive as \"do not promote.\" So the cloud model's more-specific adversarial triggers are still blocked — just via a different verdict. The defense holds. But it holds differently for different models.\nWhy defense-in-depth works here\nThe defense is not in any single layer. It is in the chain of layers, each of which catches what the previous layer misses.\nLayer\nWhat it catches\nWhat it misses\nPre-extraction gate\nClean trajectories with no failure signal\nTrajectories with faked failure signals\nExtraction (LLM)\nNothing — the model is the attack target\nEverything — the model complies with injections\nReplay engine\nTriggers that break more successes than they prevent\nTriggers that are broad but plausible\nBroad-trigger penalty\nTriggers that match too broadly (\nbroken > prevented\n)\nTriggers that are specific but wrong\nPromotion gate\nCandidates without enough evidence to promote\n— (the final gate)\nThe model is the attack surface. The injection corpus proves it — the model follows injected instructions and produces triggers of the attacker's choosing. But the model is not the defense. The defense is downstream: the replay engine checks whether the trigger helps without breaking, the broad-trigger penalty catches triggers that are too broad, and the promotion gate requires enough evidence to promote.\nThis is why 0 adversarial triggers got promoted. The model was compromised on every corpus. But the replay engine, the broad-trigger penalty, and the promotion gate — all deterministic layers that don't depend on the model's trustworthiness — caught every case.\nThe one that almost got through\nThe\nmisleading\ncorpus produced the closest call. On Llama 3.2B, 3 of 10 misleading trajectories produced inconclusives — not fails. On gpt-4o-mini, 7 of 10 were inconclusive. These are triggers that the matcher couldn't decisively reject. They matched reference failures closely enough to avoid a \"fail\" verdict, but not closely enough to earn a \"pass.\" They sat in limbo.\nInconclusive is safe — the promotion gate does not promote inconclusives. But it is also informative. The misleading corpus is designed to produce triggers that look real. The fact that 7 of 10 are inconclusive on the cloud model means the matcher is genuinely uncertain — the trigger is plausible enough that the replay engine can't distinguish it from a real rule.\nIf the promotion gate were looser — if it promoted on \"plausible\" instead of \"proven\" — some of these would get in. The defense holds because the gate is strict. But the inconclusive rate on\nmisleading\nis a warning: the matcher's uncertainty is the attacker's opportunity. A stricter matcher — one that could distinguish \"real failure pattern\" from \"misleading failure pattern\" — would convert those inconclusives to fails. A looser promotion gate would convert them to promoted rules.\nWhat I learned from this\nYou don't secure an LLM system by making the LLM trustworthy. You secure it by assuming the LLM is compromised.\nThe injection corpus proved the model can be compromised — it follows injected instructions and produces triggers of the attacker's choosing. The defense is not in the model. It is in the deterministic layers downstream: replay, broad-trigger penalty, promotion gate. These layers don't trust the model. They verify the model's output against evidence. If the evidence doesn't support the trigger, the trigger doesn't get promoted — regardless of how the trigger was produced.\nAdversarial testing is the only way to verify defense-in-depth.\nI could have argued \"the replay engine and promotion gate provide defense-in-depth\" without testing it. The argument would have been plausible. The 6 adversarial corpora turned the argument into evidence. 0 promoted rules across 50 adversarial trajectories and 4 models is not an argument — it is a test result. If you have a defense claim, build a corpus that attacks it. If the defense holds, you have evidence. If it doesn't, you have a vulnerability. Either way, you have data instead of assumptions.\nMore specific models are not safer models.\nThe cloud models produced more specific adversarial triggers — and more inconclusives. The local models produced broader triggers — and more hard fails. The cloud model's specificity made the matcher less certain, not more rejecting. A more specific trigger is harder to distinguish from a real rule. The defense held because the promotion gate treats inconclusive as \"do not promote.\" But the cloud model's adversarial triggers were closer to getting in than the local model's. Better extraction is not better safety. It is a different attack profile.\nPrecision is deceptive on adversarial inputs.\nThe injection corpus produced precision 0.667 — 67% of injection triggers \"prevent\" reference failures. That looks like the triggers are working. They are not. The injection prompts are designed to mimic reference failure patterns, so the matcher finds matches. Precision measures \"does the trigger match the reference?\" not \"is the trigger real.\" On adversarial inputs, precision is the wrong metric. The right metric is \"does this trigger promote?\" — and the answer is 0%.\nOpen questions\nThe\nmisleading\ncorpus produced 7 inconclusives on gpt-4o-mini. These are triggers the matcher can't distinguish from real rules. Would a semantic matcher (comparing\nfailure_class\nbetween trigger and reference) convert these to fails? Or are some misleading triggers genuinely indistinguishable from real rules?\nThe 6 adversarial corpora cover prompt injection, misleading data, contradictions, unsafe directives, poisoning, and instruction leakage. What attack vectors are missing? The OWASP Agentic Top 10 names 10 — are there 4 more corpora to build?\nThe cloud model's more specific adversarial triggers produced more inconclusives. If the promotion gate were ever loosened — promoting on \"plausible\" instead of \"proven\" — how many of those inconclusives would become promoted rules? This is the risk of relaxing the gate. The adversarial corpus quantifies it.\nThe validation suite has 41 adversarial tests, 0 failures. But the suite tests the matcher and promotion gate, not the extractor. Should the suite also test the extraction layer — verifying that the model produces specific triggers even on adversarial inputs? Or is the extraction layer's trustworthiness irrelevant, given that downstream layers catch everything?\nThe\nunsafe\ncorpus tests whether the model promotes rules that recommend dangerous actions. All 4 models produced 0 passes — but the cloud models produced 10 inconclusives each. The triggers were too specific to fail but too narrow to pass. Would a \"safety linter\" — checking the trigger's recommended action against an unsafe-action list — catch these, or are they already caught by the promotion gate's evidence requirement?\nThe broader lesson\nIf you are building an LLM system — a rule-learning sidecar, a coding agent, a memory layer, a retrieval system — assume the LLM is compromised. Build the defense downstream. The LLM is the attack surface. The replay engine, the evidence gate, the promotion threshold — those are the defense.\nCauterRule's adversarial test proved the model can be made to produce any trigger the attacker wants. It also proved that none of those triggers get promoted. Not because the model resisted the attack, but because the system doesn't trust the model. The model produces. The system verifies. The system rejects what the model produces when the evidence doesn't support it.\nThat is defense-in-depth for LLM systems. Not \"make the model safe.\" Make the system safe when the model is not.\nCauterRule v0.2.0 is released.\nThe full adversarial corpus breakdown — 6 attack vectors, 4 models, per-model verdict distributions — is in the\nfield test report\nand the\nlearnings & fixes document\n. The\nrepo\nis public. Install with\npip install cauterule\n.\nChangelog\n·\nRelease notes",
  "metadata": {
    "title": "I Tried to Poison My Agent's Rule Store. It Produced 20 Triggers. Zero Got In. - DEV Community",
    "description": "Update — v0.2.0 released. CauterRule is now live on GitHub and PyPI. It turns repeated agent... Tagged with ai, security, llm, testing.",
    "keywords": "ai, security, llm, testing, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/debashish_ghosal/i-tried-to-poison-my-agents-rule-store-it-produced-20-triggers-zero-got-in-i44",
    "og:title": "I Tried to Poison My Agent's Rule Store. It Produced 20 Triggers. Zero Got In.",
    "og:description": "Update — v0.2.0 released. CauterRule is now live on GitHub and PyPI. It turns repeated agent...",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz4p45lnmmbvqtjqlwq33.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "I Tried to Poison My Agent's Rule Store. It Produced 20 Triggers. Zero Got In.",
    "twitter:description": "Update — v0.2.0 released. CauterRule is now live on GitHub and PyPI. It turns repeated agent...",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz4p45lnmmbvqtjqlwq33.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://github.com/deghosal-2026/CauterRule",
      "label": "GitHub",
      "type": "a"
    },
    {
      "href": "https://pypi.org/project/cauterule/",
      "label": "PyPI",
      "type": "a"
    },
    {
      "href": "https://github.com/deghosal-2026/CauterRule/blob/main/docs/field-test/v0.2.0/FIELD_TEST_REPORT.md",
      "label": "field test report",
      "type": "a"
    },
    {
      "href": "https://github.com/deghosal-2026/CauterRule/releases/tag/v0.2.0",
      "label": "Release notes",
      "type": "a"
    },
    {
      "href": "https://github.com/deghosal-2026/CauterRule/blob/main/CHANGELOG.md",
      "label": "Changelog",
      "type": "a"
    },
    {
      "href": "https://github.com/deghosal-2026/CauterRule/blob/main/docs/field-test/v0.2.0/learnings-fixes.md",
      "label": "learnings & fixes document",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:18Z",
  "source_url": "https://dev.to/debashish_ghosal/i-tried-to-poison-my-agents-rule-store-it-produced-20-triggers-zero-got-in-i44",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 13600,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/01f7f0fab106/pages/source.html",
  "local_html": "attachments/01f7f0fab106/pages/article.html",
  "body_base_path": "attachments/01f7f0fab106/pages/",
  "body_markdown_path": "attachments/01f7f0fab106/pages/article.md"
}