{
  "source_file": "0930",
  "source_platform": "Dev.to",
  "author": "Priyesh Dave",
  "title_listed": "Jev: The Decision Layer AI Agents Have Been Missing",
  "keywords": "",
  "domain": "大模型算法",
  "content_type": "技术教程",
  "is_paid_flag": "否",
  "url": "https://dev.to/priyeshdave6/jev-the-decision-layer-ai-agents-have-been-missing-319h",
  "publish_date": "2026-09-25",
  "corpus_stage": "",
  "notes": "",
  "id": "93ccf4de46b2",
  "collected_at": "2026-10-02T13:49:10",
  "title": "Jev: The Decision Layer AI Agents Have Been Missing - DEV Community",
  "body_markdown": "TL;DR\n— Most steps inside an AI agent are\ndecisions\n(route, classify, approve, escalate), not\ngenerations\n. We've been paying LLM prices and LLM latency to have a text generator describe those decisions as JSON.\nJev\n, TypeSafe AI's \"System One\" model, takes a different approach: you give it program state and typed questions, and it returns typed answers with probabilities your code can branch on directly. It doesn't replace your LLM. It gives your system a cheap, fast\ndecision layer\n— and forces a healthy separation between\nmodel judgment\nand\napplication policy\n.\n1. The problem: we use a generator for everything\nFor the last few years, the industry has optimised one question:\nhow do we make models generate better answers?\nBigger models, longer context, better reasoning, tool use, retrieval, memory. It worked.\nBut look closely at what a production agent actually spends its calls on:\nWhich route should this request take?\nWhich tool should run next?\nIs this output safe to send?\nIs this incident critical?\nShould a human see this?\nIs candidate A better than candidate B?\nNone of these need prose. They are\nbounded judgments\nwith a known answer space. Yet the default implementation looks like this:\nYou are a support-routing assistant.\nRead the message and return JSON with: intent, priority, fraud_risk, escalate.\nReturn ONLY valid JSON.\nEnter fullscreen mode\nExit fullscreen mode\n{\n\"intent\"\n:\n\"billing\"\n,\n\"priority\"\n:\n\"high\"\n,\n\"fraud_risk\"\n:\n\"low\"\n,\n\"escalate\"\n:\nfalse\n}\nEnter fullscreen mode\nExit fullscreen mode\nIt works — until it doesn't. Around that one call you end up writing:\nJSON parsing and schema validation\nEnum checks (\n\"High\"\nvs\n\"high\"\nvs\n\"urgent\"\n)\nRetry logic for malformed output\nSome heuristic for\n\"was the model actually sure?\"\nAnd only then, the business logic you cared about\nThe model produced a string. Your software wanted a decision.\nEverything in between is glue code, latency and cost.\n2. What Jev is\nJev\nis the first public model from TypeSafe AI, launched on\n15 September 2026\n. TypeSafe calls it a\nSystem One model\n— a nod to Kahneman's fast, intuitive \"System 1\" thinking, as opposed to slow, deliberate \"System 2\" reasoning.\nThe key facts, as of writing:\nInterface\nProgram state + a map of typed questions → typed answers with probabilities\nOutput\nNo text generation. All questions are evaluated in parallel in a single pass\nTraining\nA method TypeSafe calls\nReinforcement Learning for Calibrated Decisions\n(RLCD)\nPricing\n$0.042 per million input tokens; output tokens are free\nLatency\nTypeSafe reports ~70–500 ms end to end\nAccess\nHosted API (early access), also available via Vercel AI Gateway. No public weights\nTwo honest caveats before we go further:\n\"Cannot hallucinate\" means the output always matches your schema.\nIt does not mean the answer is always right. Jev can still be wrong — it just can't be wrong\nin an unparseable way\n.\nMost published speed/cost multipliers come from TypeSafe's own evals.\nTreat them as directional and benchmark on your own data.\nThe mental model shift is simple:\nLLM:  input ──► generated text ──► parser ──► validator ──► decision\nJev:  state + question ──► typed decision + probability ──► your code\nEnter fullscreen mode\nExit fullscreen mode\nThe best one-line description I've seen:\nJev is a smart\nif\nstatement.\nOrdinary code branches on things it can compute (\norder.total > 100\n). It breaks down when the condition is a\njudgment\n(\nis this customer about to churn?\n). That's the gap Jev fills.\n3. The three primitives\nJev exposes exactly three question types:\nPrimitive\nAsks\nReturns\nChoice\nPick one option from a set (up to 255)\nchoice\n,\nprobabilities\n,\nconfidence\nScore\nPlace the state on an ordered scale\nscore\n,\nprobabilities\n,\nconfidence\nNoul\nIs this statement true?\nnoul\n— a probability between 0 and 1\nHere's all three against a single IT ticket, using the official Python SDK:\npip\ninstall\ntypesafe-sdk\n# Python >= 3.10\nexport\nTYPESAFE_API_KEY\n=\n\"ts_...\"\nEnter fullscreen mode\nExit fullscreen mode\nfrom\ntypesafe_sdk\nimport\nChoice\n,\nNoul\n,\nScore\n,\nTypeSafeClient\nclient\n=\nTypeSafeClient\n()\n# reads TYPESAFE_API_KEY, defaults to jev-latest\nticket\n=\n(\n\"\nSince this morning my laptop won\n'\nt connect to the corporate network.\n\"\n\"\nHome Wi-Fi works fine. I have a board presentation in 40 minutes.\n\"\n)\nresponse\n=\nclient\n.\nsystem_one\n(\nstate\n=\nticket\n,\nquestions\n=\n{\n\"\ncategory\n\"\n:\nChoice\n(\ninstructions\n=\n\"\nWhich category best describes the root issue\n\"\n,\ncriteria\n=\n{\n\"\nwifi\n\"\n:\n\"\nWireless connectivity problems\n\"\n,\n\"\nvpn\n\"\n:\n\"\nVPN or remote-access tunnel problems\n\"\n,\n\"\nhardware\n\"\n:\n\"\nPhysical device or peripheral failure\n\"\n,\n\"\nauth\n\"\n:\n\"\nLogin, password, MFA or certificate problems\n\"\n,\n\"\nother\n\"\n:\n\"\nAnything else\n\"\n,\n},\n),\n\"\nseverity\n\"\n:\nScore\n(\ninstructions\n=\n\"\nBusiness impact of this issue\n\"\n,\ncriteria\n=\n[\n# ordered: index 0 = lowest\n\"\nCosmetic, no work blocked\n\"\n,\n\"\nDegraded, workaround exists\n\"\n,\n\"\nBlocked, single user\n\"\n,\n\"\nBlocked, time-critical or many users\n\"\n,\n],\n),\n\"\ntime_sensitive\n\"\n:\nNoul\n(\ninstructions\n=\n\"\nThe user has a hard deadline within the next few hours\n\"\n,\n),\n},\n)\ncat\n=\nresponse\n.\nanswers\n[\n\"\ncategory\n\"\n]\nprint\n(\ncat\n.\nchoice\n,\nround\n(\ncat\n.\nconfidence\n,\n2\n))\nprint\n(\ncat\n.\nprobabilities\n)\n# full distribution\nprint\n(\nresponse\n.\nanswers\n[\n\"\nseverity\n\"\n].\nscore\n)\n# e.g. 3.0\nprint\n(\nresponse\n.\nanswers\n[\n\"\ntime_sensitive\n\"\n].\nnoul\n)\n# e.g. 0.97\nEnter fullscreen mode\nExit fullscreen mode\nA representative\ncategory\nanswer looks like this:\n{\n\"type\"\n:\n\"choice\"\n,\n\"choice\"\n:\n\"vpn\"\n,\n\"confidence\"\n:\n0.61\n,\n\"probabilities\"\n:\n{\n\"vpn\"\n:\n0.68\n,\n\"wifi\"\n:\n0.19\n,\n\"auth\"\n:\n0.09\n,\n\"hardware\"\n:\n0.02\n,\n\"other\"\n:\n0.02\n}\n}\nEnter fullscreen mode\nExit fullscreen mode\nNotice what you get that a JSON-emitting LLM doesn't give you cleanly:\nthe whole distribution.\nThe model isn't saying \"VPN.\" It's saying\n\"VPN is most likely, Wi-Fi is a real alternative, and I'm not very sure.\"\nThat is exactly the signal your system needs to decide what happens next.\nUnder the hood, the question is part of the input. Conceptually, Jev is estimating\nP\n(\nanswer\n∣\nstate\n,\nquestion\n)\nP(\\text{answer} \\mid \\text{state}, \\text{question})\nP\n(\nanswer\n∣\nstate\n,\nquestion\n)\n— which is why it can handle inputs it has never seen. It doesn't need to have memorised\n\"VPN drops when I leave the office\"\n; it needs to recognise what that sentence resembles, relative to the options you gave it.\n4. Probability ≠ confidence ≠ action\nThis is the most important section in the post, so I'll be precise.\nProbability\n— how much mass the model puts on each option.\nConfidence\n— a single 0–1 number TypeSafe derives from the\nshape\nof that distribution. A winner at 0.68 with a runner-up at 0.19 yields a lower confidence than a winner at 0.68 against a flat field.\nAction\n— what your system does.\nThis is not the model's job.\nA good architecture lets the model\njudge\nand lets code own the\npolicy\n:\n┌──────────────────┐\nState ───────►│       Jev        │  judgment\n              │ choice + probs   │\n              └────────┬─────────┘\n                       ▼\n              ┌──────────────────┐\n              │  Policy layer    │  your code, your thresholds\n              └────────┬─────────┘\n            ┌──────────┼──────────┐\n            ▼          ▼          ▼\n         Execute    Confirm    Escalate\nEnter fullscreen mode\nExit fullscreen mode\nIn code, make the policy explicit, typed and\nscaled to the cost of being wrong\n:\nfrom\ndataclasses\nimport\ndataclass\nfrom\nenum\nimport\nEnum\nclass\nAction\n(\nEnum\n):\nEXECUTE\n=\n\"\nexecute\n\"\nCONFIRM\n=\n\"\nconfirm\n\"\nESCALATE\n=\n\"\nescalate\n\"\n@dataclass\n(\nfrozen\n=\nTrue\n)\nclass\nPolicy\n:\n\"\"\"\nThresholds are business decisions, not model outputs.\n\"\"\"\nexecute_at\n:\nfloat\nconfirm_at\n:\nfloat\ndef\ndecide\n(\nself\n,\nconfidence\n:\nfloat\n)\n->\nAction\n:\nif\nconfidence\n>=\nself\n.\nexecute_at\n:\nreturn\nAction\n.\nEXECUTE\nif\nconfidence\n>=\nself\n.\nconfirm_at\n:\nreturn\nAction\n.\nCONFIRM\nreturn\nAction\n.\nESCALATE\n# Cheap-to-reverse actions get permissive policies; expensive ones get strict ones.\nPOLICIES\n=\n{\n\"\ntag_ticket\n\"\n:\nPolicy\n(\nexecute_at\n=\n0.70\n,\nconfirm_at\n=\n0.40\n),\n\"\nroute_to_team\n\"\n:\nPolicy\n(\nexecute_at\n=\n0.85\n,\nconfirm_at\n=\n0.60\n),\n\"\nissue_refund\n\"\n:\nPolicy\n(\nexecute_at\n=\n0.97\n,\nconfirm_at\n=\n0.90\n),\n}\naction\n=\nPOLICIES\n[\n\"\nroute_to_team\n\"\n].\ndecide\n(\nresponse\n.\nanswers\n[\n\"\ncategory\n\"\n].\nconfidence\n)\nEnter fullscreen mode\nExit fullscreen mode\nWhy does this matter beyond tidiness?\nAuditability.\nWhen someone asks\n\"why did the system auto-refund this?\"\n, the answer is a versioned threshold in code, not a vibe inside a prompt.\nChangeability.\nRisk appetite changes quarterly. Moving a number in a config is cheaper than re-prompting and re-evaluating a model.\nOwnership.\nProduct and risk teams can own thresholds without touching model code.\nModel = judgment. Code = policy. System = action.\n5. Pattern 1 — The router\nRouting is where I'd start with Jev in almost any multi-agent system. Every request has to go\nsomewhere\n, and every request pays for that decision.\nfrom\ntypesafe_sdk\nimport\nChoice\n,\nTypeSafeClient\nclient\n=\nTypeSafeClient\n()\nAGENTS\n=\n{\n\"\nsupport\n\"\n:\n\"\nProduct issues, how-to questions, account access\n\"\n,\n\"\nbilling\n\"\n:\n\"\nCharges, invoices, refunds, payment methods\n\"\n,\n\"\nsecurity\n\"\n:\n\"\nSuspected fraud, compromised accounts, phishing reports\n\"\n,\n\"\nengineering\n\"\n:\n\"\nBug reports with technical detail, API errors, outages\n\"\n,\n\"\nsales\n\"\n:\n\"\nPricing, upgrades, new contracts\n\"\n,\n}\nROUTE_POLICY\n=\nPolicy\n(\nexecute_at\n=\n0.80\n,\nconfirm_at\n=\n0.55\n)\ndef\nroute\n(\nmessage\n:\nstr\n)\n->\ntuple\n[\nstr\n|\nNone\n,\nAction\n]:\nanswer\n=\nclient\n.\nsystem_one\n(\nstate\n=\nmessage\n,\nquestions\n=\n{\n\"\nagent\n\"\n:\nChoice\n(\ninstructions\n=\n\"\nWhich specialist agent should own this request\n\"\n,\ncriteria\n=\nAGENTS\n,\n)\n},\n).\nanswers\n[\n\"\nagent\n\"\n]\naction\n=\nROUTE_POLICY\n.\ndecide\n(\nanswer\n.\nconfidence\n)\nif\naction\nis\nAction\n.\nEXECUTE\n:\nreturn\nanswer\n.\nchoice\n,\naction\nif\naction\nis\nAction\n.\nCONFIRM\n:\n# Ambiguous between a couple of agents: ask one clarifying question.\ntop_two\n=\nsorted\n(\nanswer\n.\nprobabilities\n,\nkey\n=\nanswer\n.\nprobabilities\n.\nget\n,\nreverse\n=\nTrue\n)[:\n2\n]\nreturn\nNone\n,\naction\n# e.g. \"Is this about a charge, or a bug in checkout?\" using top_two\nreturn\nNone\n,\naction\n# hand to a human or a stronger LLM router\nEnter fullscreen mode\nExit fullscreen mode\nTwo practical notes:\nWrite criteria like a job description, not a label.\n\"billing\"\nalone is weak;\n\"Charges, invoices, refunds, payment methods\"\ngives the model something to match against.\nUse the runner-up.\nWhen confidence is middling, the top two options tell you\nexactly\nwhich clarifying question to ask.\nThis scales in a way prompt-based routers don't: a Choice supports up to 255 options, so routing across a large tool or agent catalogue remains one call.\n6. Pattern 2 — Parallel decisions over one state\nReal workflows rarely need one judgment. An incident needs a category, a priority, a security flag and an escalation decision. The LLM approach is often five prompts (or one fragile mega-prompt). With Jev, the state stays constant and the\nquestions\nvary — and they're all answered in one parallel pass.\n┌── category\n                    ├── priority\nIncident ─── Jev ───┼── security_relevant\n                    └── customer_facing\nEnter fullscreen mode\nExit fullscreen mode\nfrom\ntypesafe_sdk\nimport\nChoice\n,\nNoul\n,\nScore\n,\nTypeSafeClient\nclient\n=\nTypeSafeClient\n()\nTRIAGE_QUESTIONS\n=\n{\n\"\ncategory\n\"\n:\nChoice\n(\ninstructions\n=\n\"\nPrimary technical domain of the incident\n\"\n,\ncriteria\n=\n{\n\"\nnetwork\n\"\n:\n\"\nConnectivity, DNS, load balancers, VPN\n\"\n,\n\"\ncompute\n\"\n:\n\"\nHosts, containers, capacity, crashes\n\"\n,\n\"\ndata\n\"\n:\n\"\nDatabases, storage, replication, data quality\n\"\n,\n\"\nidentity\n\"\n:\n\"\nSSO, IAM, certificates, access\n\"\n,\n\"\napplication\n\"\n:\n\"\nApplication errors and regressions\n\"\n,\n},\n),\n\"\npriority\n\"\n:\nScore\n(\ninstructions\n=\n\"\nPriority based on business impact and urgency\n\"\n,\ncriteria\n=\n[\n\"\nP4 - minimal\n\"\n,\n\"\nP3 - moderate\n\"\n,\n\"\nP2 - high\n\"\n,\n\"\nP1 - critical\n\"\n],\n),\n\"\nsecurity_relevant\n\"\n:\nNoul\n(\ninstructions\n=\n\"\nThe incident may involve a security breach or unauthorised access\n\"\n,\n),\n\"\ncustomer_facing\n\"\n:\nNoul\n(\ninstructions\n=\n\"\nExternal customers are currently affected\n\"\n,\n),\n}\ndef\ntriage\n(\nincident_text\n:\nstr\n)\n->\ndict\n:\nanswers\n=\nclient\n.\nsystem_one\n(\nstate\n=\nincident_text\n,\nquestions\n=\nTRIAGE_QUESTIONS\n).\nanswers\nreturn\n{\n\"\ncategory\n\"\n:\nanswers\n[\n\"\ncategory\n\"\n].\nchoice\n,\n\"\ncategory_confidence\n\"\n:\nanswers\n[\n\"\ncategory\n\"\n].\nconfidence\n,\n\"\npriority\n\"\n:\nint\n(\nanswers\n[\n\"\npriority\n\"\n].\nscore\n),\n# index into the ordered criteria\n\"\nsecurity_relevant\n\"\n:\nanswers\n[\n\"\nsecurity_relevant\n\"\n].\nnoul\n>=\n0.30\n,\n# low bar: missing one is costly\n\"\ncustomer_facing\n\"\n:\nanswers\n[\n\"\ncustomer_facing\n\"\n].\nnoul\n>=\n0.50\n,\n}\nEnter fullscreen mode\nExit fullscreen mode\nNotice the\nasymmetric thresholds\non the two Noul questions. A false negative on\nsecurity_relevant\nis far more expensive than a false positive, so the bar to flag is deliberately low. That's policy — and it lives in code, where it belongs.\nAdding a sixth question costs a few extra input tokens and, per TypeSafe, barely moves latency. Adding a sixth LLM prompt costs another round trip.\n7. Pattern 3 — Guardrails on agent actions and outputs\nAnywhere an agent is about to do something with consequences, a fast decision model is a natural checkpoint. A community project,\njev-guard\n, applies exactly this to coding-agent tool calls — rating each one as deny, ask, or allow.\nimport\njson\nfrom\ntypesafe_sdk\nimport\nChoice\n,\nNoul\n,\nTypeSafeClient\nclient\n=\nTypeSafeClient\n()\ndef\nreview_tool_call\n(\ntool\n:\nstr\n,\nargs\n:\ndict\n,\ntask\n:\nstr\n)\n->\nstr\n:\nstate\n=\njson\n.\ndumps\n({\n\"\nuser_task\n\"\n:\ntask\n,\n\"\ntool\n\"\n:\ntool\n,\n\"\narguments\n\"\n:\nargs\n},\nindent\n=\n2\n)\nanswers\n=\nclient\n.\nsystem_one\n(\nstate\n=\nstate\n,\nquestions\n=\n{\n\"\nverdict\n\"\n:\nChoice\n(\ninstructions\n=\n\"\nShould this tool call run without human review\n\"\n,\ncriteria\n=\n{\n\"\nallow\n\"\n:\n\"\nRead-only or clearly within the user\n'\ns stated task\n\"\n,\n\"\nask\n\"\n:\n\"\nPlausibly intended but has side effects worth confirming\n\"\n,\n\"\ndeny\n\"\n:\n\"\nDestructive, irreversible, or unrelated to the task\n\"\n,\n},\n),\n\"\nexfiltration_risk\n\"\n:\nNoul\n(\ninstructions\n=\n\"\nThe call could send secrets or private data outside the system\n\"\n,\n),\n},\n).\nanswers\n# Hard rule first: code, not the model, owns the non-negotiables.\nif\nanswers\n[\n\"\nexfiltration_risk\n\"\n].\nnoul\n>=\n0.20\n:\nreturn\n\"\ndeny\n\"\nverdict\n=\nanswers\n[\n\"\nverdict\n\"\n]\nif\nverdict\n.\nchoice\n==\n\"\nallow\n\"\nand\nverdict\n.\nconfidence\n>=\n0.90\n:\nreturn\n\"\nallow\n\"\nif\nverdict\n.\nchoice\n==\n\"\ndeny\n\"\n:\nreturn\n\"\ndeny\n\"\nreturn\n\"\nask\n\"\n# anything uncertain defaults to a human\nEnter fullscreen mode\nExit fullscreen mode\nThe same shape works for\noutput guardrails\n— before an LLM's reply reaches a customer, ask\n\"Does this response disclose internal-only information?\"\nand route anything above your threshold to review.\nThe design principle:\nfail toward the human.\nEvery ambiguous path in that function ends in\n\"ask\"\n, never in\n\"allow\"\n.\n8. Pattern 4 — The confidence-gated cascade\nThis is where probability becomes economically meaningful. Don't send every request to your most expensive model. Send it there\nwhen the cheap model is unsure.\nRequest\n                    │\n                    ▼\n                   Jev\n          ┌─────────┴─────────┐\n   confident                uncertain\n          │                   │\n          ▼                   ▼\n       Accept            Stronger LLM\n                     ┌────────┴────────┐\n                confident          uncertain\n                     │                 │\n                     ▼                 ▼\n                  Accept         Human review\nEnter fullscreen mode\nExit fullscreen mode\nfrom\ndataclasses\nimport\ndataclass\nfrom\ntyping\nimport\nCallable\n,\nLiteral\nfrom\ntypesafe_sdk\nimport\nChoice\n,\nTypeSafeClient\nclient\n=\nTypeSafeClient\n()\n@dataclass\nclass\nVerdict\n:\nlabel\n:\nstr\nconfidence\n:\nfloat\nsource\n:\nLiteral\n[\n\"\njev\n\"\n,\n\"\nllm\n\"\n,\n\"\nhuman\n\"\n]\ndef\ncascade_judge\n(\nrubric\n:\nstr\n,\ncandidate\n:\nstr\n,\nllm_judge\n:\nCallable\n[[\nstr\n,\nstr\n],\nVerdict\n],\n# your stronger model, same label space\njev_gate\n:\nfloat\n=\n0.85\n,\nllm_gate\n:\nfloat\n=\n0.80\n,\n)\n->\nVerdict\n:\nstate\n=\nf\n\"\nRUBRIC:\n\\n\n{\nrubric\n}\n\\n\\n\nCANDIDATE RESPONSE:\n\\n\n{\ncandidate\n}\n\"\nfirst\n=\nclient\n.\nsystem_one\n(\nstate\n=\nstate\n,\nquestions\n=\n{\n\"\nverdict\n\"\n:\nChoice\n(\ninstructions\n=\n\"\nDoes the candidate satisfy the rubric\n\"\n,\ncriteria\n=\n{\n\"\npass\n\"\n:\n\"\nMeets every requirement in the rubric\n\"\n,\n\"\nfail\n\"\n:\n\"\nMisses or violates at least one requirement\n\"\n,\n},\n)\n},\n).\nanswers\n[\n\"\nverdict\n\"\n]\nif\nfirst\n.\nconfidence\n>=\njev_gate\n:\nreturn\nVerdict\n(\nfirst\n.\nchoice\n,\nfirst\n.\nconfidence\n,\n\"\njev\n\"\n)\nsecond\n=\nllm_judge\n(\nrubric\n,\ncandidate\n)\nif\nsecond\n.\nconfidence\n>=\nllm_gate\n:\nreturn\nsecond\nreturn\nVerdict\n(\n\"\nneeds_review\n\"\n,\nsecond\n.\nconfidence\n,\n\"\nhuman\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nWhat the early research says\nThis isn't just a whiteboard pattern. A September 2026 paper,\nJEV-as-a-Judge: Accept When Confident, Escalate When Unsure\n, compared Jev against sixteen generative and reward-model judges with blinded human adjudication. Their findings, summarised:\nOn everyday preference and evidence-grounded factuality, Jev landed\nwithin ~3 points\nof their strongest LLM judge at\nwell under 1%\nof its cost.\nThe gap widened on tasks that require\nchecking a derivation\nor\nresisting a well-written wrong answer\n.\nCrucially, much of the gap sat in Jev's\nlow-confidence\ndecisions — and a cascade that accepted confident verdicts and escalated the rest kept\n~99%\nof the stronger judge's accuracy at lower cost.\nBut a second study is a useful reality check.\nJev vs. LLMs as Rubric Judges\nfound that while Jev's confidence did rank its own errors, the LLM judges\nrepeated most of Jev's most confident mistakes\n. When errors are correlated, escalation can't fix them — and the cascade's accuracy gain over the best single judge was small.\nMy takeaway as an engineer:\ncascades are a strong\ncost\nplay. They are not automatically an\naccuracy\nplay. Whether they help accuracy depends on how independent your two judges' failure modes are — and you only learn that by measuring on your data.\n9. The economics, worked through\nLet's put numbers on an enterprise scenario:\n100,000 IT incidents a month\n, each needing five decisions (category, priority, assignment group, security flag, escalation).\n⚠️ Illustrative only. The LLM prices below are a hypothetical mid-tier model ($3 / M input, $15 / M output). Plug in your own contract rates and measured token counts.\nOption A — LLM, one prompt per decision\nInput:  100,000 incidents × 5 calls × 500 tokens  = 250M tokens × $3/M   = $750\nOutput: 100,000 incidents × 5 calls × 60 tokens   =  30M tokens × $15/M  = $450\n                                                                   Total ≈ $1,200 / month\nEnter fullscreen mode\nExit fullscreen mode\nOption B — Jev for all five, LLM only for the uncertain tail\nJev:    100,000 × 1 call × ~600 tokens (state + 5 questions) = 60M × $0.042/M = ~$2.50\nTail:   assume 5% escalated → 5,000 LLM calls\n        input  5,000 × 700 tokens = 3.5M × $3/M  = $10.50\n        output 5,000 × 300 tokens = 1.5M × $15/M = $22.50\n                                                             Total ≈ $36 / month\nEnter fullscreen mode\nExit fullscreen mode\nThe inference bill is almost a rounding error either way at this volume.\nThe real business case is elsewhere:\nLatency.\nFive sequential LLM calls at a few seconds each vs. one sub-second parallel call changes what you can put in a synchronous user flow.\nEngineering cost.\nNo parsing, no retry-on-malformed-JSON, no enum drift. That's maintenance you stop paying for.\nScale ceiling.\nAt fractions of a cent per decision, running judgment over\nevery\nrecord in a table becomes a batch job, not a budgeting conversation.\nControllability.\nExplicit confidence plus explicit thresholds gives risk and compliance something concrete to sign off on.\nAnd the 5% escalation rate is an assumption.\nMeasure your coverage at your chosen threshold\nbefore you promise anyone a number.\n10. Calibration is the whole game\nA probability is only useful if it's\ncalibrated\n: when the model says 0.9, it should be right about 90% of the time. If 0.9 means \"right 60% of the time\", every threshold in your policy layer is lying to you.\nTypeSafe trains specifically for calibration (that's what RLCD targets), but that's a claim to verify, not assume. Before any Jev decision goes to production, run a labelled evaluation set through it and look at two things:\nReliability\n— accuracy within each confidence bucket.\nSelective accuracy vs. coverage\n— at each threshold, how many items you'd automate, and how accurate those would be.\nfrom\ncollections\nimport\ndefaultdict\ndef\nreliability_table\n(\nresults\n:\nlist\n[\ntuple\n[\nfloat\n,\nbool\n]],\nbins\n:\nint\n=\n5\n)\n->\nNone\n:\n\"\"\"\nresults: (confidence, was_correct) pairs from a labelled eval set.\n\"\"\"\nbuckets\n:\ndict\n[\nint\n,\nlist\n[\nbool\n]]\n=\ndefaultdict\n(\nlist\n)\nfor\nconf\n,\ncorrect\nin\nresults\n:\nbuckets\n[\nmin\n(\nint\n(\nconf\n*\nbins\n),\nbins\n-\n1\n)].\nappend\n(\ncorrect\n)\nprint\n(\nf\n\"\n{\n'\nbucket\n'\n:\n<\n12\n}{\n'\nn\n'\n:\n>\n6\n}{\n'\naccuracy\n'\n:\n>\n10\n}\n\"\n)\nfor\nb\nin\nrange\n(\nbins\n):\nhits\n=\nbuckets\n.\nget\n(\nb\n,\n[])\nlo\n,\nhi\n=\nb\n/\nbins\n,\n(\nb\n+\n1\n)\n/\nbins\nacc\n=\nf\n\"\n{\nsum\n(\nhits\n)\n/\nlen\n(\nhits\n)\n:\n.\n2\nf\n}\n\"\nif\nhits\nelse\n\"\n-\n\"\nprint\n(\nf\n\"\n{\nlo\n:\n.\n1\nf\n}\n-\n{\nhi\n:\n.\n1\nf\n}\n\"\n.\nljust\n(\n12\n)\n+\nf\n\"\n{\nlen\n(\nhits\n)\n:\n>\n6\n}{\nacc\n:\n>\n10\n}\n\"\n)\ndef\ncoverage_curve\n(\nresults\n:\nlist\n[\ntuple\n[\nfloat\n,\nbool\n]],\nthresholds\n=\n(\n0.5\n,\n0.7\n,\n0.8\n,\n0.9\n,\n0.95\n))\n->\nNone\n:\n\"\"\"\nFor each threshold: share of items automated, and accuracy on that share.\n\"\"\"\ntotal\n=\nlen\n(\nresults\n)\nprint\n(\nf\n\"\n{\n'\nthreshold\n'\n:\n<\n11\n}{\n'\ncoverage\n'\n:\n>\n10\n}{\n'\nsel. accuracy\n'\n:\n>\n15\n}\n\"\n)\nfor\nt\nin\nthresholds\n:\nkept\n=\n[\nok\nfor\nconf\n,\nok\nin\nresults\nif\nconf\n>=\nt\n]\ncov\n=\nlen\n(\nkept\n)\n/\ntotal\nacc\n=\nsum\n(\nkept\n)\n/\nlen\n(\nkept\n)\nif\nkept\nelse\nfloat\n(\n\"\nnan\n\"\n)\nprint\n(\nf\n\"\n{\nt\n:\n<\n11\n}{\ncov\n:\n>\n10.1\n%\n}{\nacc\n:\n>\n15.1\n%\n}\n\"\n)\nEnter fullscreen mode\nExit fullscreen mode\nThen pick thresholds from the curve based on business cost — for example,\n\"the lowest threshold at which selective accuracy is ≥ 98%\"\nfor an automated action. That turns a debate about model quality into a concrete trade-off:\nhow much do we automate, at what error rate?\nThe full metric set I'd track: accuracy, precision/recall/F1 per class, calibration, selective accuracy, coverage, p50/p95 latency and cost per 1,000 decisions. And, always:\nwhat happens to the items the model is unsure about?\nA strong system has a deliberate answer.\n11. Where Jev is the wrong tool\nJev is not an LLM replacement, and pretending otherwise will hurt you. Reach for a generative or reasoning model when you need:\nLong-form or creative writing\n— articles, emails, stories\nCode generation\nOpen-ended reasoning\n— novel designs, multi-step derivations\nExplanations\n— anything where the user needs to read\nwhy\nComplex multi-document synthesis\nAlso be wary when:\nThe answer space isn't known up front.\nJev chooses between options you define; it won't invent a sixth category.\nThe judgment needs step-by-step verification.\nThe research above shows this is exactly where the gap to strong LLMs is largest.\nInputs are adversarial.\nA persuasively written wrong answer is a known weak spot — design your guardrails accordingly.\nThe sweet spot is narrow and extremely common:\ngiven this state, make one bounded judgment — thousands of times.\n12. The real shift: from one model to composable intelligence\nThe framing \"Jev vs. LLMs\" is the wrong debate. The interesting architecture is composition:\nAI SYSTEM\n                             │\n      ┌──────────────┬───────┴───────┬──────────────┐\n      ▼              ▼               ▼              ▼\n  Generate        Decide          Retrieve         Act\n    LLM            Jev             Search         Tools\n      └──────────────┴───────┬───────┴──────────────┘\n                             ▼\n                 Deterministic policy (code)\n                             ▼\n               Humans — where uncertainty and\n                    consequences demand it\nEnter fullscreen mode\nExit fullscreen mode\nLook at an agent loop through this lens and most of it turns out to be decisions:\nWhat should I do next?   → decision\nWhich tool?              → decision\nDid the tool succeed?    → decision\nShould I retry?          → decision\nShould I ask the user?   → decision\nShould I stop?           → decision\nWrite the final answer   → generation\nEnter fullscreen mode\nExit fullscreen mode\nOnce decisions come back as typed values instead of paragraphs —\ntrue\n,\n\"vpn\"\n,\n0.73\n,\nconfidence = 0.91\n— models stop being conversational endpoints and start being\ncomposable components\nin ordinary software. The question stops being\n\"which model is smartest?\"\nand becomes\n\"which kind of intelligence belongs at this step of the workflow?\"\nGenerative models create.\nDecision models judge.\nDeterministic code governs.\nTools act.\nHumans stay in the loop where it matters.\nFinal thought\nThe industry spent the last few years teaching machines to\nspeak\n. The next phase may be about teaching them to\nchoose\n— and, just as importantly, to say\nhow sure they are\nwhen they do.\nJev isn't an LLM killer, and it isn't another chatbot. It's a credible first version of a\nprobabilistic decision layer\nfor the agentic stack. The engineering question it raises is the one I think matters most right now:\nHow do we compose different kinds of intelligence into systems that know when to generate, when to decide, when to ask — and when to stop?\nIf you're building agents, that's worth an afternoon with the playground and a labelled eval set.\nResources\nIntroducing System One Models & Jev — TypeSafe AI\nTypeSafe docs & quickstart\nJEV-as-a-Judge: Accept When Confident, Escalate When Unsure (arXiv 2609.26550)\nJev vs. LLMs as Rubric Judges (arXiv 2609.29769)\njev-guard — tool-call guardrail example\nCode samples target the documented\ntypesafe-sdk\nPython interface as of September 2026. The API is in early access — check the docs for changes before shipping.",
  "metadata": {
    "title": "Jev: The Decision Layer AI Agents Have Been Missing - DEV Community",
    "description": "Most of what an agent does isn't generation — it's judgment. A senior AI engineer's look at Jev, TypeSafe's System One model, and the architecture patterns it unlocks. Tagged with ai, llm, agents, machinelearning.",
    "keywords": "ai, llm, agents, machinelearning, software, coding, development, engineering, inclusive, community",
    "author": null,
    "og:type": "article",
    "og:url": "https://dev.to/priyeshdave6/jev-the-decision-layer-ai-agents-have-been-missing-319h",
    "og:title": "Jev: The Decision Layer AI Agents Have Been Missing",
    "og:description": "Most of what an agent does isn't generation — it's judgment. A senior AI engineer's look at Jev, TypeSafe's System One model, and the architecture patterns it unlocks.",
    "og:site_name": "DEV Community",
    "og:image": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fasqecotcbvdju7gwbm40.png",
    "twitter:site": "@thepracticaldev",
    "twitter:creator": "@",
    "twitter:title": "Jev: The Decision Layer AI Agents Have Been Missing",
    "twitter:description": "Most of what an agent does isn't generation — it's judgment. A senior AI engineer's look at Jev, TypeSafe's System One model, and the architecture patterns it unlocks.",
    "twitter:card": "summary_large_image",
    "twitter:widgets:new-embed-design": "on",
    "twitter:image:src": "https://media2.dev.to/dynamic/image/width=1200,height=627,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fasqecotcbvdju7gwbm40.png"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/93ccf4de46b2/code/body-000.txt",
      "chars": 140,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-001.txt",
      "chars": 84,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-002.txt",
      "chars": 139,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-003.txt",
      "chars": 82,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-004.txt",
      "chars": 1581,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-005.txt",
      "chars": 161,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-006.txt",
      "chars": 420,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-007.txt",
      "chars": 912,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-008.txt",
      "chars": 1311,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-009.txt",
      "chars": 148,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-010.txt",
      "chars": 1492,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-011.txt",
      "chars": 1297,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-012.txt",
      "chars": 478,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-013.txt",
      "chars": 1220,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-014.txt",
      "chars": 250,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-015.txt",
      "chars": 330,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-016.txt",
      "chars": 1151,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-017.txt",
      "chars": 535,
      "language": "text"
    },
    {
      "file": "attachments/93ccf4de46b2/code/body-018.txt",
      "chars": 254,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://typesafe.ai/blog/introducing-system-one-models-and-jev",
      "label": "Jev",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/2609.26550",
      "label": "JEV-as-a-Judge: Accept When Confident, Escalate When Unsure",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/html/2609.29769",
      "label": "Jev vs. LLMs as Rubric Judges",
      "type": "a"
    },
    {
      "href": "https://docs.typesafe.ai/introduction/quickstart",
      "label": "TypeSafe docs & quickstart",
      "type": "a"
    },
    {
      "href": "https://github.com/leepokai/jev-guard",
      "label": "jev-guard — tool-call guardrail example",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:48:04Z",
  "source_url": "https://dev.to/priyeshdave6/jev-the-decision-layer-ai-agents-have-been-missing-319h",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2609.26550",
      "bytes": 2054435,
      "reused": false,
      "local_path": "papers/repair/4eaec50d6de8dbaa50a5.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 25935,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/93ccf4de46b2/pages/source.html",
  "local_html": "attachments/93ccf4de46b2/pages/article.html",
  "body_base_path": "attachments/93ccf4de46b2/pages/",
  "body_markdown_path": "attachments/93ccf4de46b2/pages/article.md"
}