parallelConstraintDecoding

# Parallel Constrained Decoding

Fill a whole JSON schema of booleans and enums in **two forward passes**
instead of generating it token by token — with calibrated confidence per
field and output that is schema-valid by construction.

[](https://www.youtube.com/watch?v=C4Vjbf_HvYY)

*Watch the demo (YouTube, Devoxx): the web app races the parallel engine
against grammar-constrained token-by-token generation on the same model.*

This repository contains:

- **`java/`** — a Java implementation of the technique that drives
  [llama.cpp](https://github.com/ggml-org/llama.cpp) directly through the JDK
  Foreign Function & Memory API (no JNI). It is the fastest engine measured
  here, and it ships with a local **web app** that races it against a
  grammar-constrained token-by-token baseline on the same model, lets you
  build your own extraction scenarios, and charts benchmarks. It can load any
  GGUF, including the models of a local Ollama install.
- **`python/`** — a port of the original engine published at
  [harshatheg/Qwen-2.5-1B-RLCD](https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD)
  (MLX, Apple Silicon only), kept for reference and comparison. Despite the
  name, that Hugging Face repo contains no weights and no fine-tune — it is
  source code running stock `Qwen2.5-1.5B-Instruct` with a custom decoding
  strategy. See [NOTICE](NOTICE) for what was vendored and changed.
- **`presets/`** — extraction scenarios shared by both engines (fraud
  triage, code-security audit, support triage, a 255-choice tariff router,
  spam/phishing triage, and Devoxx CFP talk routing with a bank of 100 real
  talks).

## Benchmark summary (Apple M1 Ultra, GPU otherwise idle, median after warm-up)

| Preset | Fields | Token-by-token JSON (Python, MLX) | Python parallel (MLX) | **Java na