jev-eval-mcp

# jev-eval-mcp

An eval-first MCP server for [TypeSafe's Jev](https://docs.typesafe.ai), a System One model that
returns typed judgments (`noul`, `choice`, `score`) with probabilities instead of generated text.

Most Jev integrations expose "ask the model a question". The hard part in practice is not asking —
it is knowing whether a question works and where to put the threshold. This server is built around
that:

| tool | use it for |
| --- | --- |
| `jev_ask` | prototype a question against one state and see the full probability distribution |
| `jev_map` | run a question set over many items, get one compact row each, filter/sort/save |
| `jev_eval` | measure question variants against labeled examples: accuracy, calibration, threshold sweep, worst misses |

`jev_map` exists because the expensive part of triaging 500 files, findings, or tickets is reading
them into the agent's context. `jev_eval` exists because a threshold picked by vibes is the usual
reason a classifier gate misbehaves in production.

## Install

Requires Node 20+ and a TypeSafe API key from https://console.typesafe.ai/settings/keys.

Register the server with your MCP client. Claude Code:

```bash
claude mcp add jev -e TYPESAFE_API_KEY=sk-... -- npx -y jev-eval-mcp
```

Or in a `mcp.json`-style config:

```json
{
  "mcpServers": {
    "jev": {
      "command": "npx",
      "args": ["-y", "jev-eval-mcp"],
      "env": { "TYPESAFE_API_KEY": "sk-..." }
    }
  }
}
```

For [opencode](https://opencode.ai), in `~/.config/opencode/opencode.json` (or a
project-level `opencode.json`):

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "jev": {
      "type": "local",
      "command": ["npx", "-y", "jev-eval-mcp"],
      "enabled": true,
      "environment": { "TYPESAFE_API_KEY": "{env:TYPE