A Jev PR Check as a GitHub Action

Code review bots have a reputation problem: they write paragraphs nobody reads. An AI PR review action built on Jev, TypeSafe AI's decision model, does the opposite. It asks a handful of closed questions about the diff and posts a pass, warn, or needs-review status. No essays, just verdicts a reviewer can act on in two seconds.

Where decision models fit across a dev workflow (test triage, log classification, agent compaction) is covered in Jev for developers. This page builds one specific thing: an automated PR check.

What already exists

You're not first. A community project, jev-review-action, is a GitHub Action that reviews and classifies PRs with Jev alone (build). DiffJury takes a public PR link and says whether it's safe to merge or needs review (build). And a "Magic Jev Ball" browser tool checks CI, diff size, and existing reviews, then Jev decides in about 200 milliseconds, as reported (build).

Read their source before writing your own; borrowing a working question is faster than inventing one. The 200 ms figure is builder-reported, and there are no official Jev benchmarks.

The recipe

1. Decide what the check is for. Pick questions a senior reviewer asks in the first ten seconds:

   • "Does this diff change authentication, permissions, or secrets handling? YES / NO / UNCLEAR"
   • "Does this diff modify a database migration or schema? YES / NO / UNCLEAR"
   • "Does the PR description match what the diff actually changes? YES / NO / UNCLEAR"
   • "Does this diff add or change tests for the code it modifies? YES / NO / UNCLEAR"

   These aren't "is this code good". They're routing signals for reviewer attention.

2. Keep the payload small. Send the PR title, description, and a trimmed diff: file paths plus changed hunks. Skip lockfiles and generated code. Check docs.typesafe.ai for current input limits rather than guessing.

3. Write the workflow. Trigger on pull request opened and synchronized events. The job fetches the diff, runs the questions, and reports results.

# pseudocode, not real API syntax or a real workflow file
on pull_request (opened, synchronize):
    diff    = git_diff(base, head, exclude=["*.lock", "dist/"])
    context = pack(pr.title, pr.body, diff)
    results = [jev.choose(q, context, ["YES", "NO", "UNCLEAR"]) for q in PR_QUESTIONS]
    status  = summarize(results)            # pass / warn / needs-review
    post_check_run(status, table_of(results))
    add_labels_for(results)                 # e.g. "touches-auth"

Per ecosystem documentation, the model is typesafe-ai/jev via the Vercel AI Gateway, returning per-choice probabilities. Store the gateway key as a repository secret, never in the workflow file.

4. Report as a check, not a comment. A status check with a small table (question, verdict, confidence) is scannable and doesn't spam the conversation. Labels like "touches-auth" or "schema-change" help route the PR to the right reviewer.

5. Start non-blocking. Run the check as informational for two weeks. Compare its flags against what reviewers actually cared about. Then decide which questions, if any, become required.

Should it be a merge gate?

The obvious objection: "If it's cheap and fast, why not block merges on it?" Because a merge is the kind of action you don't want triggered or prevented by a single verdict. A reasonable ladder:

• Warn on low-confidence or UNCLEAR answers.
• Require a human approval (not a bot pass) when high-risk questions come back YES, such as auth or migrations.
• Never auto-merge on a Jev pass alone. Auto-merge should depend on tests, required reviews, and branch protection, with the verdict as one input at most.

That's the same "claims become questions, questions don't become authority" logic from AI agent verification. If agents are opening your PRs, this check becomes part of that verification layer.

Automated PR checks that stay useful

Tune questions, not thresholds, first. If a question flags constantly, it's probably vague. "Does this touch security?" flags everything; "Does this change how passwords, tokens, or API keys are read or stored?" doesn't.

Version the questions in the repo. Keep them in a config file alongside the workflow so changes get reviewed like code.

Don't ask it to explain the bug. Jev returns decisions, not prose. If you want a written review, that's a chat model's job, and you can trigger one only when the verdicts say the PR is risky.

Frequently asked questions

Can Jev write code review comments?

No, Jev returns decisions rather than generated text. Use it to decide which PRs need attention and have a chat model or a human write the comments.

Should an AI PR check block merges?

Start it as informational, then require human approval when high-risk questions return YES. A lone verdict should never be the merge gate, as AI agent verification explains.

How fast is a Jev PR check?

One builder reports a merge decision in about 200 milliseconds, as reported, so most of your workflow time will be checkout and diff fetching. Your own numbers depend on diff size and question count.

Are there existing Jev GitHub Actions I can use?

Yes, community projects like jev-review-action exist, and Jev for developers surveys more of the dev-workflow builds. Review any third-party action's source before giving it repository secrets.