jeff

# jeff

A self-hosted implementation of TypeSafe's [jev System One API](https://docs.typesafe.ai/api),
powered by [GLiFormer](https://huggingface.co/knowledgator/gliformer-large-v1) (400M parameters).
Use the official `typesafe-sdk` by pointing `TYPESAFE_BASE_URL` at jeff.

Supports `choice` (pick an option), `score` (rate on ordered levels), and `noul` (probability of yes).
Cheaper to self-host, but less accurate than jev on reasoning-heavy tasks. See [benchmarks](#benchmarks).

[Quickstart](#quickstart) · [Deploy](#deploy-on-modal) · [Configuration](#configuration) · [API](#api-and-compatibility) · [Development](#development)

## Quickstart

Requires [uv](https://docs.astral.sh/uv/) and Python 3.12. Run from the repository root:

```bash
uv sync --extra dev
uv run hf download knowledgator/gliformer-large-v1 --local-dir models/gliformer-large-v1
JEFF_API_KEYS=devkey uv run jeff
```

Serves at `http://localhost:8000`. Device selection: CUDA → MPS → CPU.

The sync above also installs `typesafe-sdk`. Save this as `example.py`:

```python
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient(api_key="devkey", base_url="http://localhost:8000")
result = client.system_one(
    "I was charged twice. Please help ASAP.",
    {
        "billing": Noul(instructions="Is this about billing?"),
        "tone": Choice(
            instructions="What is the tone?",
            criteria={"calm": None, "angry": "hostile"},
        ),
        "urgency": Score(
            instructions="How urgent is this?", criteria=["low", "medium", "high"]
        ),
    },
)
print(result.nouls["billing"].noul)
print(result.choices["tone"].choice)
print(result.scores["urgency"].score)
```

In another terminal:

```bash
uv run python example.py
```

For an existing SDK app