Jev's Architecture, Unmasked
A ~28-minute forensic essay reconstructing how Jev — TypeSafe's "System One" decision model — is almost certainly built, based on ~10,000 black-box API probes (1,029 instrumented probe records, 6,800 benchmark records, a downloadable evidence bundle) after TypeSafe declined to share its research. The framing: ordinary LLMs generate confidence as text ("90% confident"), and software then acts on that unvalidated token sequence; Jev instead reads decision probabilities directly from its internal representations, trained against outcomes (RLCD — Reinforcement Learning for Calibrated Decisions), and returns all of a request's answer distributions in parallel without generating text. The author dismisses the X-thread takes ("12 million views for a JSON classifier? we're in a bubble") as missing the actual point.
The reconstruction: a causal transformer (likely a sparse mixture-of-experts) repurposed for decisions, ending after prefill with a direct probability readout instead of a decode loop. A small head maps the final hidden state to logits over K allowed answers — either fixed option slots or a pointer-style scorer over each option's own representation (the essay leaves this deliberately undecided). One shared state KV prefix serves many isolated question branches, packed into a single sequence (≤32,768 tokens per branch, ≤65,536 per request with the state counted once); prior art exists in Hydragen (shared-prefix attention) and DeFT (tree-structured attention). Parallelism describes scheduling and lack of answer dependencies, not one GPU per question.
The evidence is carefully labelled published / observed / inferred. Key experiments: a secret-code intervention shows a declaration in a sibling question is invisible (p = 0.00) while the same text in the shared state is visible (0.90–0.92) — behavioural isolation plus shared-state access. Token accounting is exactly additive, and output_tokens is exposed as a post-hoc billing figure (4 shared tokens + 15 per answer + identifier length) that matches none of 192 public tokenizers and does not track latency — evidence about billing, not about generation. A reference-card experiment shows the decision can use information placed after the option list (16/16 correct with the card last), so options are read together rather than scored independently. Adding an irrelevant fifth option shifted the odds between existing options (+0.38 → +0.11 log-odds, replicated across ten randomised blocks), falsifying fixed independent logits plus unchanged softmax — listwise processing. Injected fake options never displaced real ones: option boundaries are marked in a way text cannot forge. Reversing option order moved a classification probability from ~0.84–0.89 to ~0.93–0.96, an operational hazard for any deployed threshold.
On calibration: RLCD-style training targets proper scoring rules (log loss, Brier); ten-bin expected calibration error was 0.0313 on a 1,200-item MMLU sample (990 of 1,200 predictions in the 0.9–1.0 bin); fresh generated math showed 86.7% accuracy at 0.83 average top probability but 32% at 0.30 on two-step word problems. The API's confidence field is a normalisation formula, not a second learned estimate. Sparse MoE is the least certain component but the likeliest backbone: prefill-only serving is compute-bound, and MoE's decode-time serving costs mostly vanish (~30k tokens in ~160 ms; a dense 70B on 8×H100 would need ~1 s). The final ranking of commitments: direct probability outputs (published), question isolation and option-order effects (observed), KV sharing / causal attention / readout choice / sparse experts (progressively more inferred).