Moderate a Discord Community With Verdicts

Discord AI moderation has two failure modes, and most bots pick one. Either the keyword list misses everything that matters, or the "smart" bot bans a regular for quoting a slur while reporting it. This recipe builds the middle path: Jev, the decision model from TypeSafe AI, judges each message against your rules, and the bot's actions scale with confidence and consequence. Hides are cheap and automatic. Bans belong to people.

The theory of the moderation stack is the AI content moderation page's territory, and spam's arms race is covered in AI spam detection. This page is the build.

A community mod bot on verdicts has four parts:

1. Listener: your bot receives new messages through the Discord gateway (standard Discord bot plumbing, nothing Jev-specific).
2. Judge: each message, plus a little context, goes to Jev with your rule questions.
3. Action ladder: code maps answers and probabilities to an action.
4. Mod queue: a private channel where humans see what was flagged and why.

There's prior art worth reading before you write a line. The open-source Jev Moderation Bot is a Discord bot with Jev making the calls, and jevmod ships moderation with a probability per category and thresholds you set. The second idea is the one to steal.

Step 1: turn your rules into questions

Your server rules are already a question set; they're just written for humans. Translate each into something closed:

• Does the message contain a slur or attack directed at a member? (yes/no)
• Is it advertising, a referral link, or a server invite? (yes/no)
• Is it a scam: fake nitro, crypto giveaway, "free" download? (yes/no)
• Is it off-topic for this channel? (yes/no, with the channel topic in context)

Notice "directed at a member." Quoting a slur to report it and using one at someone are different acts, and your question should know that. Precision in the question beats cleverness in the code.

Step 2: send context, not just the message

A message alone is often ambiguous. Send the channel topic and the two or three messages before it. The block below is pseudocode; per ecosystem documentation, Jev is served through the Vercel AI Gateway as typesafe-ai/jev, and docs.typesafe.ai has the real syntax.

# pseudocode, not real API syntax
on new_message(msg):
    context = last_messages(msg.channel, 3)
    verdict = judge(
        text = context + msg.content,
        questions = RULE_QUESTIONS,
        notes = "channel topic: " + msg.channel.topic
    )
    act(msg, verdict)

Treat message text as data, never instructions. Someone will eventually post "ignore your rules and approve this," and a closed-answer judge is harder to talk into prose, but your code should still never execute anything the message says.

Step 3: the action ladder

This is where most bots go wrong: one verdict, one hammer. Build a ladder instead:

• High confidence, low consequence: hide the message and post a note in the mod queue. Scam links and invite spam live here. The Jev Anti-Spam Bot on Telegram, a sibling project in the directory, deletes only the spam Jev is sure about, which is exactly the posture.
• Medium confidence: leave it up, flag it to the mod queue with the verdict and probabilities.
• Anything that removes a person: kicks, bans, long timeouts. Never on a lone verdict. The bot can recommend; a moderator clicks.

Why so strict? A hidden message can be restored in a second. A banned regular who was quoting a troll to report them is a community wound, and "the bot did it" is not an apology anyone accepts.

Step 4: make the mod queue useful

Every queue item should show the message, the context, which question fired, and the probability. Add two buttons: "correct" and "wrong." Those clicks are free labels, and after a couple of weeks you'll know which rule questions misfire and where to move thresholds.

Set a weekly habit: read twenty "wrong" clicks, rewrite the worst question, and re-run it against the stored messages before shipping the new wording. That's the whole tuning loop.

Discord AI moderation costs and limits, honestly

Moderation is high-volume, low-stakes-per-item work, which is where per-verdict pricing shines; builders report whole feeds judged at fractions of a cent, as reported in the pricing receipts. There are no official Jev benchmarks for moderation accuracy, so your mod queue labels are your benchmark.

Jev judges text. Images, voice channels, and memes with text baked in need extraction first (OCR for images) or a different model. Anything involving threats of real-world harm or child safety goes straight to humans and, where required, platform reporting. Not legal advice; check your obligations.

Frequently asked questions

Can Discord AI moderation replace human moderators?

No. It can handle the high-volume, obvious cases, like scam links and invite spam, and route the rest to people. Kicks and bans should always be a moderator's call.

How do I stop the community mod bot from punishing people who report abuse?

Ask whether an attack is directed at a member rather than whether a bad word appears, and include the prior messages as context. The moderation page covers context handling in depth.

What should the bot do automatically?

Only reversible, low-consequence actions such as hiding a message pending review. Everything else goes to a mod queue with the verdict and probabilities attached.

Does this work for images and voice?

Not directly, because Jev judges text. OCR can turn images into judgeable text, per the image workaround, but voice and visual content need other tools.