Which phrases make a paragraph read like AI? I built a heatmap and measured it on HC3

Description and use

AiSniff highlights AI‑style phrases with rule explanations and rewrite hints, yet overall detection performance is modest.

Problem

Identify specific phrases that make a paragraph appear AI‑generated rather than relying on a single opaque score.

Outcome

AiSniff provides phrase‑level AI‑writing pattern detection with rule explanations and rewrite hints, but its overall detection performance is weak (AUROC 0.63 English, 0.59 Chinese).

Workflow

• Paste text into the web interface or run the CLI tool.
• AiSniff scans the text against 61 versioned JSON rules for English and Traditional Chinese.
• Underlined phrases are displayed with rule details and rewrite suggestions.
• A side panel groups hits by type, shows density per 1,000 words or characters, and offers a before/after compare view.
• Export the marked text as PNG if needed.
• In CI, run npx github:Arthur031221/AiSniff lint draft.md to fail a build when density is too high.

Evidence

I built AiSniff.

Source

It runs fully in the browser. There is no server and no model call, so nothing you paste leaves the page.

Source

There are 61 rules in versioned JSON packs for English and Traditional Chinese.

Source

At least one English rule fires on 28% of ChatGPT answers and 2% of human answers.

Source

The AUROC is 0.63 for English and 0.59 for Chinese, and 72% of the English ChatGPT answers have no hit at all.

Source