jevchat

# jevchat

We know [Jev](https://docs.typesafe.ai/api).

`jevchat` turns that into a chat model. At every step it asks Jev one question:

> Given the user's question and the reply written so far, which symbol comes next?

The options are an alphabet plus an option to stop emitting. Jev returns a probability for each
one, and the sampler draws the next symbol from that normalised distribution.
Append, repeat, and stop when STOP is drawn.

There are several alphabets (including truncated token lists) and sampling strategies available.  

The idea is for fun, the cost is somewhat impractical, and the results are hilarious.

<img width="450" height="403" alt="image" src="https://github.com/user-attachments/assets/8be3a47a-da3c-4df6-a617-d485c61a7db7" />


This was a Claude accelerated experiment.  I described the sampling algorithms, strategies, and so on, and it implemented them.

## Setup

```bash
poetry install
```

Put your Jev key in `.env` next to `pyproject.toml` (git-ignored):

```
api_key="..."
```

`JEV_API_KEY` and `TYPESAFE_API_KEY` are also accepted. Values in `.env` win over
exported ones, so editing the file is enough to switch keys.

## Use

```bash
poetry run jevchat                          # interactive chat
poetry run jevchat ask "do people need water?"
poetry run jevchat alphabets                # what you can sample from
poetry run jevchat bench                    # compare every mode (table below)
```

The reply appears as it is sampled, in a panel with a live readout of the
generation rate — symbols/s, characters/s, milliseconds per API call, elapsed
time — and the top few symbols Jev scored at the last step, so you can watch the
distribution the sampler is drawing from.

**Ctrl-C cancels.** The first press stops generation once the in-flight request