jevmlx

# jevmlx

[](https://github.com/bnsd55/jevmlx/actions/workflows/ci.yml) [](https://github.com/bnsd55/jevmlx/actions/workflows/build.yml) [](LICENSE) [](pyproject.toml)

jevmlx turns a schema of fields (booleans, enums, multi-selects) and a context string into a single batched forward pass on a local Apple Silicon model. Every allowed option for every field is scored from logits in one prefill — no text generation — and the JSON is assembled from the winners, with a probability per field.

## Quickstart

```bash
git clone https://github.com/bnsd55/jevmlx && cd jevmlx && ./setup.sh
jevmlx decide --preset support_triage --json          # one decision, CLI
jevmlx serve --model fast --port 8000                  # HTTP server
curl -s localhost:8000/decide -d '{"schema":{"x":{"type":"enum","choices":["a","b"],"description":"d"}},"context":"pick one"}'
```

Requires an Apple Silicon Mac (M1+) and Python 3.12+. First use of a model alias downloads weights (~2 GB `fast`, ~4.5 GB `quality`).

## Install

```bash
pip install git+https://github.com/bnsd55/jevmlx          # library
uv tool install git+https://github.com/bnsd55/jevmlx     # CLI only
git clone https://github.com/bnsd55/jevmlx && cd jevmlx && ./setup.sh   # dev
```

### Model aliases

| Alias | Resolves to | Use |
|---|---|---|
| `quality` | `mlx-community/Qwen2.5-7B-Instruct-4bit` | **default** — best accuracy |
| `fast` | `mlx-community/Qwen2.5-3B-Instruct-4bit` | lower latency |
| `test` | `mlx-community/Qwen2.5-1.5B-Instruct-4bit` | tests only (too small for production) |

A full Hub id also works (`--model mlx-community/Llama-3.2-3B-Instruct-4bit`).

## Why not structured output

Structured output asks the model to write the JSON, token by token, then parses it and retries on failure. When the answer set is known a