Which phrases make a paragraph read like AI? I built a heatmap and measured it on HC3

Описание и применение

AiSniff выявляет фразы, которые делают текст похожим на написанный ИИ, показывая правила и подсказки для переписывания, но общая точность низкая.

Задача

Identify specific phrases that make a paragraph appear AI‑generated rather than relying on a single opaque score.

Результат

AiSniff provides phrase‑level AI‑writing pattern detection with rule explanations and rewrite hints, but its overall detection performance is weak (AUROC 0.63 English, 0.59 Chinese).

Рабочий процесс

• Paste text into the web interface or run the CLI tool.
• AiSniff scans the text against 61 versioned JSON rules for English and Traditional Chinese.
• Underlined phrases are displayed with rule details and rewrite suggestions.
• A side panel groups hits by type, shows density per 1,000 words or characters, and offers a before/after compare view.
• Export the marked text as PNG if needed.
• In CI, run npx github:Arthur031221/AiSniff lint draft.md to fail a build when density is too high.

Доказательства

I built AiSniff.

Источник

It runs fully in the browser. There is no server and no model call, so nothing you paste leaves the page.

Источник

There are 61 rules in versioned JSON packs for English and Traditional Chinese.

Источник

At least one English rule fires on 28% of ChatGPT answers and 2% of human answers.

Источник

The AUROC is 0.63 for English and 0.59 for Chinese, and 72% of the English ChatGPT answers have no hit at all.

Источник