shipwithjev

Catalog / Research & data

0068GitHub

OpenJev Verdict 2.0

A 151M non-autoregressive decision engine, with its numbers published.

The second version of the open Verdict engine. It is a 151 million parameter non-autoregressive decision model, and the repository reports 77.10% accuracy, a Brier score of 0.0636 and an expected calibration error of 0.0144 on their typed-decisions set, which it says beats Jev and Laya.

Heman10x-NGU/openJev-verdict-2.0README ↗
# openJev-verdict-2.0: Non-Autoregressive System 1 Decision Engine

[](https://huggingface.co/heman10x/openJev-verdict-2.0)
[](https://github.com/Heman10x-NGU/openJev-verdict-2.0)
[](#the-openjev-verdict-20-benchmark-breakthrough)
[](#dual-channel-calibration)
[](#single-pass-efficiency)
[](#in-browser-webgpu-engine)
[](LICENSE)

---

## Two models in this project

This repository contains code and references for two distinct models:

1. Verdict (the 151M model evaluated on JevBench): General-purpose decision model based on ModernBERT-base and GLiClass. The public checkpoint is hosted at [heman10x/rlcd-modernbert-151m](https://huggingface.co/heman10x/rlcd-modernbert-151m), now updated to v1.4 via inference engine fixes.
2. Verdict 2.0: Specialized architecture for typed software workflows evaluated on `LocalLLaMA/typed-decisions`. The model weights are currently tracked via Git LFS pointers in `artifacts/verdict2-base/model.pt`. The benchmark numbers reported in the breakthrough section below were produced directly by `verdict2/evaluate.py` against its audited test receipt (`reports/verdict2_base_test.json`).

---

## What changed in the inference engine

These are inference fixes, not a retrain. The weights are byte-identical to the published checkpoint. Measured on the 231 public JevBench tasks.

<p align="center">
  <img src="assets/v1.4/benchmark-leaderboard-chart.png" alt="JevBench Score Leaderboard: openJev Verdict v1.4" width="100%">
</p>

<p align="center">
  <img src="assets/v1.4/benchmark-leaderboard-table.png" alt="JevBench Detailed Evaluation Table: openJev Verdict v1.4" width="100%">
</p>

The update addresses three defects in the inference engine:

1. Calibrator auto-loading and removal of the 5-option scope restriction: The engine previously failed to loa

Also filed under Research & data

  1. 0607

    Verify: new-hire onboarding completion

    An agent reports onboarding done; the judge verifies access and equipment claims.

    everyai-com · Research & data

  2. 0606

    Triage: vague meeting request gets a disposition

    A vendor asks for 30 minutes with no agenda; the judge picks the disposition.

    everyai-com · Research & data

  3. 0605

    Triage: data-loss bug gets a severity

    A note-taking app silently drops edits on flaky networks; the judge grades severity.

    everyai-com · Research & data

  4. 0604

    Triage: crash report routing + reproducibility

    A crash report with steps and logs; the judge routes it and checks reproducibility.

    everyai-com · Research & data