System 1 Models: How AI Decides Without Writing a Word

In my last article, we watched the industry fight the $O(n^2)$ cost of attention. This time the villain is different. It isn't how the model reads. It's how it answers.

Here is a task that millions of production systems run every day:

"Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."

Which department? How urgent? Is the customer about to leave?

Since LLMs arrived, the default for many teams has been to send this to an LLM with a JSON schema and wait. The model reads the ticket, then writes {"department": "billing", ...} one token at a time. Then your code parses it, validates it, and retries when the model decides "Billing Dept." is a perfectly good enum value.

We are asking a novelist to fill in a checkbox.

This month, a new category went mainstream. On September 15, TypeSafe launched Jev, which they call a System One model. Within days, the open-source Laya project answered it, from a developer who says he published the core idea a year earlier. Neither writes anything. They read the input once and return typed decisions with probabilities attached.

Most of the coverage is about speed and price. This article is about the how: what happens inside a model that decides without generating, how it learns to be honest about its confidence, and where it quietly fails.

System 1 vs System 2

Or: "The host doesn't write an essay about each table."

Daniel Kahneman split human thinking into two modes. System 1 is fast and automatic: you recognize a face, you hear the anger in a voice. System 2 is slow and deliberate: you do long division, you plan a trip.

Think of a busy restaurant. The host glances at a group of six at the door and instantly knows: table 12, the round one by the window. That's System 1. The chef designing next season's menu is System 2.

Chat LLMs are built as System 2 machines. Even for a snap judgment, they produce their answer the only way they know how: by writing it out. System 1 models are the host. They look once and point.

The Autoregression Tax

Or: "Why writing 'billing' costs more than knowing 'billing'."

An autoregressive model produces one token per forward pass. Each new token depends on all the previous ones, so the passes can't run in parallel.

A tiny structured answer like {"dept": "billing", "urg": 2} is around a dozen tokens. That means a dozen sequential trips through the entire network, and then a parser that can fail.

Figure: The same decision, made two ways. Left: one forward pass per token, then parsing. Right: one pass, every option scored at once.

Here's the naive "LLM as classifier" pattern many teams start with:

import json

ALLOWED = {"billing", "technical", "sales", "other"}

def classify_with_llm(llm, ticket, max_retries=3):
    prompt = f"""Classify this ticket. Reply ONLY with JSON:
{{"department": one of {sorted(ALLOWED)}, "urgency": 0-2, "churn_risk": true/false}}

Ticket: {ticket}"""
    for attempt in range(max_retries):
        raw = llm.generate(prompt)            # N sequential forward passes
        try:
            out = json.loads(raw)             # can fail: prose, markdown fences...
            if out["department"] in ALLOWED:  # can fail: "Billing Dept."
                return out
        except (json.JSONDecodeError, KeyError):
            pass
    raise ValueError("LLM never produced a valid answer")

To be fair, that loop is the worst case. Modern APIs offer constrained decoding and structured-output modes that guarantee valid JSON and legal enum values, so parse failures are mostly a solved problem. You can even classify with a decoder in a single pass: put the labels in the prompt and read the log-probabilities of each label's first token.

So the honest case for System 1 models isn't "LLMs can't do this." It's these:

• Cost. A decoder LLM with billions of parameters is doing a job that an encoder with a few hundred million parameters can handle, and you pay for every one of them on every ticket.
• Latency. Even the single-pass logprob trick means running that big model on every request, and a big model is a slow model.
• Engineering friction. The first-token trick breaks when labels share a first token (card_arrival_delayed vs card_arrival_status). You can fix that by scoring each full label sequence, but that means extra computation per label and more code to maintain.
• Calibration. Instruction-tuned chat models are known to lose calibration during post-training, so their token probabilities are a shaky basis for "act if confidence ≥ 0.9".

A System 1 model is built around these. The answer space is the list of options you pass in, the model is small, and its probabilities are trained to mean something.

Anatomy of a Single-Pass Decision

Or: "Put every candidate answer in the room at the same time."

The trick is to stop asking the model to produce an answer and instead ask it to score candidates that are already in its input.

Laya packs everything into a single sequence for an encoder: the question first, then every option, then the state (your ticket, email or JSON) in whatever room is left. Each option starts with a [MASK] token, which acts as a marker for "read this option here". The encoder is bidirectional, like BERT: every token can see every other token. So while it reads the ticket, the word billing in the option list is already attending to "billed twice" and "refund".

After one pass, the vector at each option's [MASK] marker has "read" the entire context. A small head turns each of those vectors into one number (a logit), and a softmax turns the logits into probabilities.

Using [MASK] as the marker is a neat fit: BERT-style encoders are pretrained to fill in [MASK] positions from the surrounding context, so that's exactly the kind of vector they're good at building.

Figure: Logits become probabilities through softmax. Temperature controls how sharp the distribution is, which will matter a lot later.

Let's build a simplified version, based on Laya's build_sequence() and DecisionModel in laya/common.py. I've stripped out the trimming logic and the extra layers so the idea stays visible.

First, packing the input and remembering where each option lives. Here's what that looks like before we write it:

Figure: Question, options and state are tokenized into one row, in that order. The position of each option's [MASK] marker is recorded, then one pass through the model turns each marker into one probability.

And here's the same thing in code:

def pack(tok, question, options, state, max_len=1024):
    """Simplified from build_sequence() in laya/common.py.

    Layout: [CLS] question [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state [SEP]
    Returns the token ids and the position of each option's [MASK] marker.
    tok: a Hugging Face tokenizer.  options: dict[label, description]
    """
    enc = lambda text: tok(text, add_special_tokens=False)["input_ids"]

    ids = [tok.cls_token_id] + enc(question) + [tok.sep_token_id]
    markers = []
    for label, desc in options.items():
        markers.append(len(ids))                  # the model reads this option HERE
        ids += [tok.mask_token_id] + enc(f" {label}: {desc}")
    ids.append(tok.sep_token_id)

    room = max_len - len(ids) - 1                 # whatever is left goes to the state
    ids += enc(state)[:room] + [tok.sep_token_id]
    return ids, markers

Then the model: an encoder, a lookup of the vector at each marker, and a layer that scores each one.

import torch
import torch.nn as nn
import torch.nn.functional as F

class ToyDecisionModel(nn.Module):
    def __init__(self, vocab_size, d=256, layers=4, heads=4, max_len=1024):
        super().__init__()
        self.tok = nn.Embedding(vocab_size, d)
        self.pos = nn.Embedding(max_len, d)
        layer = nn.TransformerEncoderLayer(d, heads, 4 * d, batch_first=True)
        self.encoder = nn.TransformerEncoder(layer, layers)  # bidirectional: no causal mask
        self.score = nn.Linear(d, 1)                         # option vector -> one logit

    def forward(self, input_ids, markers):
        # input_ids: (1, L) for a single state/question, to keep it readable
        positions = torch.arange(input_ids.size(1))
        h = self.tok(input_ids) + self.pos(positions)
        h = self.encoder(h)[0]                               # (L, d): ONE forward pass

        # read each option at its [MASK] marker position
        option_vecs = h[torch.tensor(markers)]               # (K, d)
        return self.score(option_vecs).squeeze(-1)           # (K,) logits

That's the whole trick: no decoder, no loop, no sampling. One pass, K numbers out.

The real DecisionModel adds three things on top: a learned embedding for the question type (choice picks one option, score rates on an ordinal scale, and noul answers a yes/no question with the probability of yes), a small 2-layer transformer head over the encoder output, and a small MLP instead of a single linear scorer. The data flow is the same.

Three primitives from one set of logits

Laya (and Jev) expose three question types. With the logits in hand, they're just three ways to read a distribution:

def choice(logits, T=1.0):
    """Pick one option. Returns the full distribution."""
    return F.softmax(logits / T, dim=-1)

def score(logits, T=1.0):
    """Ordinal rubric, e.g. ["not urgent", "soon", "blocking"].
    Returns the expected level, so 1.84 means 'mostly blocking'."""
    p = F.softmax(logits / T, dim=-1)
    levels = torch.arange(len(p), dtype=p.dtype)
    return (p * levels).sum()

def noul(logits, T=1.0):
    """Yes/no. Two slots in [false, true] order; return P(true)."""
    return F.softmax(logits / T, dim=-1)[1]

The score type explains a line you'll see in Laya's documentation, where urgency comes back as 1.84 / 2.0. That's not a rounding bug. It's the expected value over the three levels.

Multiple questions don't need multiple conversations either. The state is encoded alongside each question, and the questions are batched into the same forward pass. In Laya's published benchmarks for laya-multilingual on a T4 GPU, one question takes 32.8 ms, while 10 questions batched together take 72.3 ms in total, about 7 ms each. Batching amortizes the fixed cost of a forward pass. (The English laya checkpoint is slower: 39.5 ms and 158.6 ms.)

Connecting to the last article: Laya's English checkpoint is built on ModernBERT, which alternates full global attention with sliding-window local attention layers. It's the same speed-versus-reach trade-off we saw in the hybrid architectures. It's still attention, but it runs once, not once per output token.

Training a Model to Be Honest

Or: "Accuracy rewards guessing. Proper scoring rules reward knowing."

Here's the subtle part, and it's what both projects' training story centers on.

If you only reward a model for getting the argmax right, you teach it nothing about confidence. A model that says 51% and a model that says 99% get the same reward when they're right. So nothing stops the probabilities from drifting into meaningless territory.

Jev and Laya both describe their training as RLCD: Reinforcement Learning for Calibrated Decisions. The core idea is to use a strictly proper scoring rule as the reward. A scoring rule is "strictly proper" when the only way to get the best expected score is to report your true belief.

You can check this in a few lines. Suppose an event really happens 70% of the time. What happens when a model reports different probabilities?

import numpy as np

p_true = 0.7   # the event really happens 70% of the time

def expected_brier(q, p):           # E[(q - y)^2], lower is better
    return p * (1 - q) ** 2 + (1 - p) * q ** 2

def expected_logloss(q, p):         # E[-log P(y)], lower is better
    return -(p * np.log(q) + (1 - p) * np.log(1 - q))

def expected_accuracy_reward(q, p): # 1 if argmax right, else 0, higher is better
    return p if q >= 0.5 else 1 - p

for q in [0.5, 0.6, 0.7, 0.8, 0.95]:
    print(f"report {q:.2f} | Brier {expected_brier(q, p_true):.3f} "
          f"| log loss {expected_logloss(q, p_true):.3f} "
          f"| accuracy reward {expected_accuracy_reward(q, p_true):.2f}")

Output:

report 0.50 | Brier 0.250 | log loss 0.693 | accuracy reward 0.70
report 0.60 | Brier 0.220 | log loss 0.632 | accuracy reward 0.70
report 0.70 | Brier 0.210 | log loss 0.611 | accuracy reward 0.70
report 0.80 | Brier 0.220 | log loss 0.639 | accuracy reward 0.70
report 0.95 | Brier 0.273 | log loss 0.935 | accuracy reward 0.70

Both proper scores bottom out at exactly 0.70, the honest answer. Overclaiming to 0.95 is punished hard. The accuracy reward, meanwhile, is flat: it can't tell 0.6 from 0.95.

So what does the RL add?

Here's the catch: ordinary supervised training with cross-entropy already is the log score, which is strictly proper. So what is RL doing in RLCD?

Laya's fine-tuning notebook answers this concretely. Keep in mind that it only describes Laya: Jev is closed, its training recipe isn't public, and it may work quite differently despite sharing the RLCD name. The targets aren't hard labels but a teacher's soft probability distributions, and the loss has two parts:

1. Plain soft cross-entropy against those teacher distributions, kept at full weight.
2. An RL term. For each example, it samples a group of noisy variations of the model's logits, scores each variation with a reward, and pushes the model toward the variations that scored better than the group average. That group-relative comparison is the "GRPO-style" part.

The interesting part is the reward. It combines the log score with two other proper scoring rules: a spherical score, and for score questions, a ranked probability score (RPS). RPS knows that the levels are ordered, which cross-entropy doesn't:

import numpy as np

levels = ["not urgent", "soon", "blocking"]
target = np.array([0.0, 0.0, 1.0])            # the ticket really is "blocking"

near_miss = np.array([0.2, 0.7, 0.1])          # wrong, but one level off
far_miss  = np.array([0.7, 0.2, 0.1])          # wrong, two levels off

def log_score(q, t):                           # what cross-entropy optimizes
    return -(t * np.log(q)).sum()

def rps(q, t):                                 # ranked probability score (lower is better)
    return ((np.cumsum(q) - np.cumsum(t)) ** 2).sum() / (len(q) - 1)

for name, q in [("near miss", near_miss), ("far miss", far_miss)]:
    print(f"{name:9} | log loss {log_score(q, target):.3f} | RPS {rps(q, target):.3f}")

near miss | log loss 2.303 | RPS 0.425
far miss  | log loss 2.303 | RPS 0.650

Cross-entropy only looks at the probability of the right answer, so it scores both mistakes the same. RPS punishes calling a blocking ticket "not urgent" much harder than calling it "soon", which is exactly what you want from an urgency score.

One honest engineering note: all three of these rewards are differentiable. So, by the argument above, the RL machinery isn't strictly needed here: you could add the spherical score and RPS as ordinary loss terms next to cross-entropy and optimize the same objectives directly. I haven't seen an ablation comparing the two, so whether RL helps in practice here is an open question. RL earns its keep when the reward isn't differentiable, like F1 or a human judgment, and the notebook's setup would let you swap in a reward like that.

Calibration: Does 90% Mean 90%?

Or: "Trust, but measure."

Training with proper rules pushes toward honest probabilities, but it doesn't guarantee them. Laya is refreshingly upfront here: its documentation says both checkpoints are over-confident as shipped.

That's not a contradiction, and it's normal for neural networks. A proper scoring rule rewards honesty on the training data. A large network can keep lowering its training loss by growing more and more confident on examples it has effectively memorized, and that confidence carries over to new inputs where it hasn't been earned.

The standard way to measure this is Expected Calibration Error (ECE). Group predictions into confidence bins, and in each bin compare "how confident was the model" with "how often was it right". A perfectly calibrated model sits on the diagonal.

Figure: A reliability diagram. Over-confident models sit below the diagonal. One fitted temperature pulls them back toward it.

The fix is almost embarrassingly simple: temperature scaling. Divide every logit by a single number T, fitted on held-out data. Because dividing by a positive number doesn't change which logit is biggest, the top answer for choice and noul questions doesn't change, only its confidence does.

Two caveats. First, anything that thresholds a probability will change, like "act if P(yes) ≥ 0.8". That's the point of calibrating, but re-check your thresholds afterwards. Second, score answers do change: the expected level is an average over all levels, and softening the distribution pulls it toward the middle of the scale. If you round or threshold urgency scores, recalibrating changes decisions.

Here's a self-contained demo on a simulated over-confident classifier:

import numpy as np
rng = np.random.default_rng(0)

# Simulate an over-confident 4-way classifier: the model stretches
# the "true" logits by 2.5x before the softmax.
n, k = 5000, 4
true_logits = rng.normal(0, 1.2, size=(n, k))
true_probs = np.exp(true_logits) / np.exp(true_logits).sum(1, keepdims=True)
labels = np.array([rng.choice(k, p=p) for p in true_probs])
model_logits = 2.5 * true_logits

def softmax(z, T=1.0):
    z = z / T
    z = z - z.max(1, keepdims=True)
    e = np.exp(z)
    return e / e.sum(1, keepdims=True)

def ece(probs, labels, n_bins=10):
    conf = probs.max(1)
    correct = probs.argmax(1) == labels
    edges = np.linspace(0, 1, n_bins + 1)
    total = 0.0
    for lo, hi in zip(edges[:-1], edges[1:]):
        m = (conf > lo) & (conf <= hi)
        if m.any():
            total += m.mean() * abs(correct[m].mean() - conf[m].mean())
    return total

def nll(probs, labels):
    return -np.log(probs[np.arange(len(labels)), labels] + 1e-12).mean()

# Fit ONE scalar T on a held-out split by minimising NLL
fit, test = slice(0, 2500), slice(2500, None)
grid = np.linspace(0.5, 5.0, 91)
best_T = min(grid, key=lambda T: nll(softmax(model_logits[fit], T), labels[fit]))

print(f"fitted T = {best_T:.2f}")
print(f"ECE before: {ece(softmax(model_logits[test]), labels[test]):.3f}")
print(f"ECE after : {ece(softmax(model_logits[test], best_T), labels[test]):.3f}")

fitted T = 2.45
ECE before: 0.222
ECE after : 0.024

It recovers a temperature close to the 2.5 we planted, and ECE drops by an order of magnitude. On real data, the project's own experiments show the same effect: refitting temperatures moves mean ECE from 0.466 to 0.081 on the English checkpoint. It fits one temperature per question type and option count, not one global number.

Why does this matter so much? Because calibrated confidence is what lets you build the most useful pattern in this whole category, confidence gating:

if answer_confidence >= THRESHOLD:     # chosen on YOUR held-out data
    act_automatically(decision)        # fast path: System 1
else:
    escalate(decision)                 # slow path: a human, or a System 2 LLM

Where It Breaks

Or: "Every fast shortcut has a blind spot."

Both projects publish benchmarks. What makes Laya especially useful for learning is that its documentation covers its own failures in detail. Here are the ones every engineer should understand.

1. The option budget

All options have to fit in the input at the same time, so the question and options share a fixed slice of the sequence called head_max_len: 256 tokens on Laya's multilingual and typed-decisions checkpoints (192 on the English one). Whatever the head doesn't use goes to the state.

When the options fit, nothing happens: each one keeps its text (up to 48 tokens). When they overflow, build_sequence() trims every option to the same length, keeping its [MASK] marker:

def tokens_per_option(n_options, head_max_len=256):
    # applied only when the options overflow the head budget; the marker is included
    return max(4, (head_max_len - 16) // n_options)

for n in (20, 77):
    per = tokens_per_option(n)
    print(f"{n} options -> {per} tokens each ({per - 1} of text), {n * per} tokens total")

20 options -> 12 tokens each (11 of text), 240 tokens total
77 options -> 4 tokens each (3 of text), 308 tokens total

Notice the second line: 308 tokens is more than the 256-token budget. The floor of 4 tokens per option means the head spills over its budget, the question text is cut down to 8 tokens, and the state loses room too.

Figure: Options share a fixed head budget. Four options fit with room to spare, 20 get trimmed, and 77 overflow the budget while each keeps only 3 tokens of text.

With 3 tokens of text, card_arrival_delayed and card_arrival_status both become card_arrival…. This is likely a large part of why Laya scores 0.425 on Banking77 while Jev, which supports up to 255 options, scores 0.870. The two numbers aren't perfectly comparable, though: Laya was measured on all 77 intents, while Jev's score comes from a 100-example independent pilot on a 72-label version of the dataset. The documentation suggests three fixes: raise the budget, split the question into coarse and fine steps, or shortlist the top k labels with embeddings first (predict_shortlist).

2. Confidently wrong, in the wrong language

The English checkpoint is trained on English. Give it Khmer and it scores 0.000 accuracy at 95.2% confidence. Calibration can't save you here: the model has no idea it's out of its depth, so confidence gating waves every wrong answer through.

Laya's answer is architectural. A Router detects the script and language before the forward pass, in under a millisecond of pure Python, and sends non-English text to the multilingual checkpoint. It's a lesson that goes beyond this project: you can't gate on confidence for inputs the model was never trained to recognize as strange.

3. Labels that leak

Because options are read as text, the words in them matter. The documentation warns that choice options labelled true/false or yes/no can make the model follow the label instead of the description. It also documents a negation failure: on a handful of negated cancellation requests (think "I don't want to cancel"), the model sometimes picked cancel_account, once with 0.9998 probability. Neutral labels like A/B help with the first problem but not the second, so the project advises validating the exact wording you serve.

4. Zero-shot isn't the product

This is the most important honest number in the project's documentation. On the typed-decisions benchmark, the base checkpoints score 0.362 and 0.352, below a simple majority-class baseline of 0.461. The fine-tuned checkpoint reaches 0.766. As the project puts it, Laya is a fast base to specialise, not a zero-shot decision engine. Budget for fine-tuning.

Why Not Just Fine-Tune BERT?

Or: "The label set is an input, not a layer."

If you have to fine-tune anyway, a fair question is why not do what we've done since 2019: take BERT, add a classification layer with one output per label, and train it.

The difference is where the labels live. In a standard classifier, the label set is baked into the output layer. Add a new department, rename one, or split "technical" into "bug" and "outage", and you retrain. Each task also needs its own model.

In a System 1 model, the labels are part of the input, written as descriptions. The same model routes support tickets in the morning and scores invoice risk in the afternoon, and adding a label means editing a dictionary, not starting a training run. Fine-tuning here teaches the model your domain and makes its judgments sharper. It doesn't lock in a label set.

There's a tension worth admitting, though. The previous section said you need to fine-tune, and fine-tuning on one domain can make a model worse at others. So "one model for everything" works best within a related family of tasks. Laya's fine-tuned checkpoint, for example, targets the four workflows of its typed-decisions benchmark: invoice processing, security incidents, customer service and agent-trace observability. According to the project's benchmark results, it scores between 0.730 and 0.804 on each. If your tasks are very different from each other, you may still end up with more than one fine-tuned model, but you won't need a new one every time a label changes.

This idea has a history, and it's worth knowing:

• Zero-shot NLI classifiers (like the Hugging Face zero-shot-classification pipeline) already treated labels as text. They phrase each label as a hypothesis ("This text is about billing") and ask an entailment model whether the input supports it. The catch is one forward pass per label: 77 labels, 77 passes.
• GLiClass from Knowledgator, inspired by the GLiNER entity-recognition models, prefixes each label with a special «LABEL» token, packs the text and all labels into one sequence, and scores every label in a single forward pass of a BERT-style encoder (DeBERTa in its main models, with ModernBERT variants too). That label-marker trick is strikingly close to Laya's [MASK] markers. Its authors even adapted a reinforcement learning method (PPO) to train it, as described in the GLiClass paper.

So the mechanism isn't new. What Jev and Laya add is the packaging: typed primitives (choice, score, noul) instead of just labels, training aimed at calibrated probabilities, routing by language, and a shared API for plugging decisions into production systems.

Hands-On: Laya in a Few Lines

Or: "Enough theory, let's route some tickets."

python -m pip install laya

from laya import Router

router = Router()  # downloads a checkpoint on first use

state = "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
questions = {
    "department": {"type": "choice", "instructions": "Which department should handle this?",
                   "criteria": {"billing": "invoices, payments, refunds",
                                "technical": "bugs, outages, system errors",
                                "other": "everything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["not urgent", "soon", "blocking"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
}

result = router.predict(state, questions)
print(result["answers"]["department"]["choice"])  # billing
print(result["answers"]["churn_risk"]["noul"])    # probability the answer is yes
print(result["routing"]["model"])                 # english

Notice what's missing: no prompt engineering around output format, no JSON parsing, no retries. Every answer is one of the values you allowed, with a probability attached.

Laya also exposes the same POST /v1/systemone wire protocol as Jev's hosted API, so a client written for one can be pointed at the other. That makes it easy to benchmark both on your own data, which is the only benchmark that really counts.

Jev vs Laya: Reading the Scoreboard Fairly

Or: "Check who ran the numbers."

The two projects are close cousins: same core idea, same three primitives, both describing their training as RLCD. The big differences are practical. Jev is a closed, hosted API priced per input token. Laya is Apache 2.0 with open weights that run on a free Colab T4.

Laya's published benchmarks report 32.8 ms for a single question on a T4 versus third-party measurements of 236–276 ms for Jev, and 0.766 versus 0.727 on the shared typed-decisions set. Three caveats matter:

• The Jev numbers weren't measured by the Laya project. The project's documentation says so directly, which means sample sizes and prompts differ. The figures aren't always consistent either: Startup Fortune noticed that the calibration error listed for Jev is 0.246 on the project's website but 0.144 on its model card.
• The latency comparison isn't like-for-like. Laya's 32.8 ms is a model already loaded on a local GPU. The Jev figures come from calling the hosted API over the internet (one of the cited benchmarks called it from France), so they include the network round trip. Laya would be faster locally either way, but not necessarily by as much as the raw numbers suggest.
• Jev leads in places, and independent testing is thin. Besides the high-cardinality case above, Jev matches the teacher's full probability distribution better (soft accuracy 0.580 vs 0.471). Most Laya numbers are self-published, and the one small independent test I found ran 78 classification cases: Laya was the fastest model, at 30 ms per case, but scored lowest (0.590), while Jev scored highest (0.974). Two details matter, though. The testers used Laya's English base checkpoint, not the fine-tuned one, so this is exactly what "Zero-shot isn't the product" predicts. And the Jev column came from the benchmark suite author's own run against the hosted API, not from the testers themselves. With 78 cases, it's no verdict either way, which is exactly why you should benchmark on your own data.

The same test also put the opening's alternative to work. A general 27B chat model, compressed to about one bit per weight, was forced through a JSON-schema enum, with its probabilities read from its logprobs. It scored 0.885, nine points above the best purpose-built open classifier in the test, but took 1.7 seconds per decision instead of 30–66 ms. That supports the latency argument. It also complicates the calibration one. The chat model's logprobs ranked every yes/no case perfectly. Good ranking isn't the same as good calibration, and the post doesn't report calibration error, but it's a reminder that my opening claim about chat-model probabilities is something to test on your own data, not assume.

There's also a dispute worth knowing about. Laya's author says the non-autoregressive decision concept is his and was published in March 2025. Reporting describes this as an allegation of uncredited prior art, not copied code, and I haven't found a public response from TypeSafe at the time of writing. Coverage is split on how strong the claim is: Startup Fortune notes that his earlier published work was a narrow sales-conversion predictor, so it supports the ingredients more than a general-purpose model like Jev. In the Hacker News discussion, commenters pushed back that the concept has older academic precursors and that TypeSafe's contribution was shipping a product. As we saw with zero-shot NLI and GLiClass, those precursors are real. Whoever gets credit for the packaging, the underlying idea of scoring candidates with an encoder predates both projects.

Conclusion: The Brain Needs Both Systems

In the last article, the answer to "which attention wins?" was hybrids. The same holds here.

System 1 models won't replace LLMs. They can't write, plan or explain. What they can do is take the thousands of small decisions inside every AI system (route this ticket, flag this prompt, pick this tool, escalate this case) and make them fast, typed and honest about their uncertainty.

The architecture I expect to see everywhere looks like a brain: a System 1 model handles the fast, confident majority, and hands the uncertain cases up to a System 2 LLM or a human. The confidence score is the bridge between them. That's why calibration matters more than raw speed: it's what makes these models usable.

But as we've seen, it isn't a property you get once and keep. The checkpoints ship over-confident, temperature scaling is a standard fix rather than a breakthrough, and inputs the model doesn't recognize as strange slip past it entirely. Calibration has to be measured and maintained on your own data.

For the AI Solution Engineer, the new question isn't "which model is smartest?" It's "which decisions deserve a writer, and which just need a host?"