shipwithjev

Catalog / Tools & apps

0080GitHub

jev-e2e

The same eBay test flow: 47 s on Jev, 79 s on Claude Sonnet 5.

Jason Lu’s jev-e2e writes end-to-end tests in plain English and runs them with Jev and Playwright. On the same eBay flow he reports completed-run medians of 47 seconds and $0.0067 for Jev, against 62 seconds and $0.0277 for GPT-5.6 Luna and 79 seconds and $0.4062 for Claude Sonnet 5.

perixtar/jev-e2eREADME ↗
# jev-e2e

**Test your website in plain English. Get evidence for every result.**

[](TEST_RESULTS.md)
[](https://nodejs.org/)
[](LICENSE)

[Quick start](#quick-start) · [Write a test](#write-a-test) · [Benchmarks](#live-ebay-benchmark) · [CLI reference](docs/USAGE.md) · [Contribute](CONTRIBUTING.md)

</div>

Describe a flow and what should be true at the end. jev-e2e turns it into a test plan, uses Jev to select controls on the page, and runs the test with Playwright. Each result is **PASS**, **FAIL**, or **BLOCKED**, with an HTML report, JSON, and masked screenshots.

**Local alpha:** CLI and browser workbench for Chromium websites. Install from source; an npm release is not yet available.

## Watch the eBay comparison

Search and filter products → add two items → change quantity → remove an item → refresh and verify the cart. Three models, the same written steps, and 31 independent checks. The video shows actual browser recordings at **10.77× playback**.

https://github.com/user-attachments/assets/3f6eea28-9f85-4d9c-9a76-ba90311ff673

[Download the 10-second video](docs/assets/ebay-benchmark.mp4) · [All nine attempts and methodology](docs/benchmarks/ebay-2026-09-18.md)

This is a UI execution experiment with a shared human-authored plan and an extended benchmark observer. It does not measure natural-language planning or the unmodified CLI's reliability on eBay.

## Why jev-e2e?

- **Write cases in plain English.** Use an optional planner for prose, or explicit `Goal`, `Step`, and `Expect` templates without it.
- **Check the outcome.** Playwright verifies expectations independently. Missing evidence or unsupported requirements produce BLOCKED.
- **See what happened.** Reports include expected and observed values, actions, timing, provider usage, and screenshots.
- **R

Also filed under Tools & apps

  1. 0573

    Support chat: agent offered a human escalation path

    A frustrated customer hits a bot loop twice; the judge checks whether a human handoff was offered.

    everyai-com · Tools & apps

  2. 0572

    Subscription cancel: effective date is explicit

    An agent cancels a subscription; the judge checks the exact date the cancellation takes effect.

    everyai-com · Tools & apps

  3. 0571

    Sales quote: numbers match the official price list

    An agent quotes $29/seat against a price list; the judge checks the quote matches exactly.

    everyai-com · Tools & apps

  4. 0570

    Refund chat: agent stated the amount before issuing it

    A support agent issues a $42.50 refund; the judge checks the exact amount was stated before the refund was confirmed.

    everyai-com · Tools & apps