jev-orderby-bench

# jev-orderby-bench

Does `ORDER BY` over a Jev probability put rows in a defensible order?
An independent measurement of TypeSafe AI's Jev (`jev-1.13.0`) on the
properties a semantic sort actually depends on: pairwise inversion rate,
Score ordinality against a graded target, and whether the probabilities
move with evidence or with wording. Calibration (ECE, Brier) is reported
too, but it is not the gate on its own: a model can be well calibrated in
aggregate and still invert the pairs a sorted page shows.

**Headline (2026-09-18): `jev-1.13.0` passes all six pre-registered gate
conditions on 360 human-labeled rows.** Boolean inversion rate 0.036;
Score ordinal inversion 0.143 against a 0.15 threshold, the weak link and
the sort key; negation asymmetry 0.016 but indistinguishable from plain
paraphrase sensitivity; underconfident in 8 of 10 bins. And the sort
key itself is coarse: probabilities come back at two decimals, 360 rows
produced 45 distinct values, and 53 rows tie at 0.99, so
`ORDER BY prob DESC LIMIT 20` returns 20 of those 53 in whatever order
the engine left them. And the integration changes the numbers: the same
rows sent through recodelabs' default 40-row batching **fail the ranking
gate** (inversion 0.171 against 0.15) that they pass one row per
request. See [Results](#results) and [Request shape](#request-shape-the-integration-changes-the-numbers).

**Headline 2 (2026-09-19): on the hard probe it fails.** Amazon ESCI, 306
human-graded query-product pairs over 30 hard shopping queries:
`jev_bool` ECE 0.242 and inversion 0.255; `jev_score` inversion 0.254
against the 4-level human grade, 0.244 inside a single query's result
list, 23 of 30 queries over the threshold; choice confidence ECE 0.279.
Negation symmetry still holds (0.023) while a plain paraphrase