shipwithjev

Catalog / Tools & apps

0012GitHub

jev-turn-analysis

Reads a finished coding session and judges each turn.

Zach Hobbs’s Rust command-line tool goes over completed Claude Code and Codex sessions and uses Jev to say what happened in each turn.

hobbs/jev-turn-analysisREADME ↗
# Jev Turn Analysis

**Analyze and improve your coding agent harness.**

`jta` reads your Claude Code and Codex sessions, finds where work went well or got
stuck, and suggests specific changes to your agent instructions, skills, and tools.
Start with whether the task succeeded and was checked. Then look at wasted effort.

## Get started: two commands

Install from this checkout with a stable Rust toolchain:

```sh
cargo install --path . --locked
```

With `~/.cargo/bin` on your `PATH`, open a project where you use Codex or Claude
Code and run:

```sh
jta init
jta analyze
```

**`jta init` gets you ready.** It walks you through the Jev API key and a report CLI
(`codex` or `claude`). Key entry is hidden, and saved keys work across all your projects.

**`jta analyze` does the work.** It finds your project's sessions, scores them
with Jev, and shows what succeeded, what was verified, and where to investigate.
Each session gets a live progress bar; completed sessions leave a short result
line. No transcript export or separate import step is needed.

Run `jta analyze` again whenever you have new sessions. Unchanged results are
reused automatically, with cached sessions clearly marked.

## Generate the full report

After analysis, turn recurring patterns into suggested edits to `AGENTS.md`,
skills, and tools:

```sh
jta report
```

Choose a report CLI during `jta init`, or set it explicitly:

```sh
jta init --review-backend codex            # Default
jta init --review-backend claude
```

Install and authenticate the selected CLI separately. **JTA manages only the Jev
API key.** Report generation uses the CLI's own authentication, with no review
API key or HTTP endpoint in JTA configuration.

`jta report` saves a **Markdown report** and a JSON companion with full-corpus
statist

Also filed under Tools & apps

  1. 0573

    Support chat: agent offered a human escalation path

    A frustrated customer hits a bot loop twice; the judge checks whether a human handoff was offered.

    everyai-com · Tools & apps

  2. 0572

    Subscription cancel: effective date is explicit

    An agent cancels a subscription; the judge checks the exact date the cancellation takes effect.

    everyai-com · Tools & apps

  3. 0571

    Sales quote: numbers match the official price list

    An agent quotes $29/seat against a price list; the judge checks the quote matches exactly.

    everyai-com · Tools & apps

  4. 0570

    Refund chat: agent stated the amount before issuing it

    A support agent issues a $42.50 refund; the judge checks the exact amount was stated before the refund was confirmed.

    everyai-com · Tools & apps