jev-curate
Streaming filter and scorer for Parquet and JSONL datasets.
# `jev-curate` **Support:** fuel the next build — [](https://buymeacoffee.com/AkashPriyadarshi) **High-Throughput Synthetic & Pretraining Dataset Sifter Powered by TypeSafe AI (Jev)** > Beta: v0.2.0 is experimental. Expect rough edges. Please contribute by opening an issue or PR. **Live:** [jev-curate.vercel.app](https://jev-curate.vercel.app) Measured: **23.9 rows/sec** single-node local mock (`bash bench/run_mock.sh`, results in `bench/results/mock.json`). Not measured: **1,500+ rows/sec** is a horizontal-scaling target, not a benchmark result. **Product Hunt:** [jev-curate Upcoming](https://www.producthunt.com/products/jev-curate?launch=jev-curate) (launches Oct 13, 12:01am PT) <a href="https://www.producthunt.com/products/jev-curate?embed=true&utm_source=badge-featured&utm_medium=badge&utm_campaign=badge-jev-curate" target="_blank" rel="noopener noreferrer"><img alt="jev-curate - Sift 1B-row datasets 444x cheaper with Jev | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1272040&theme=light&t=1791323078281"></a> [](https://pypi.org/project/jev-curate/) [](LICENSE) [](https://typesafe.ai) By **[Akash Priyadarshi](https://github.com/AkashPriyadarshii)** [Why jev-curate](#why-jev-curate) • [Quickstart](#quickstart) • [CLI Reference](#cli-reference) • [Python API](#python-api) • [Architecture](#architecture) • [Non-Goals](#non-goals) • [Ecosystem](#ecosystem) </div> [](https://github.com/AkashPriyadarshii/jev-curate/stargazers) [](https://crates.io/crates/jev-curate) [](https://crates.io/crates/jev-curate) [](https://github.com/AkashPriyadarshii/jev-curate/releases) --- ## Why `jev-curate`? Cleaning 10M to 1B rows of synthetic reasoning data, instruction tunin
