AI Writing Markers: Building an Open Dataset and Checker on Hugging Face

Description and use

The ai‑writing‑markers tool offers transparent, source‑backed analysis of text, highlighting LLM‑style patterns and enabling authors to edit drafts without relying on opaque probability scores.

Problem

Commercial AI detectors give opaque probability scores with high false‑positive rates, especially against non‑native English writers.

Outcome

The checker provides transparent, source‑backed feedback on AI‑like writing patterns, empowering users to edit drafts without relying on opaque probability scores.

Workflow

• Clone the ai‑writing‑markers repo
• Run python3 check.py <file> or pipe text to the checker
• Review the per‑category hit counts and source‑backed markers
• Edit the draft to reduce dense clusters of flagged patterns
• Re‑run the checker to confirm improvements

Evidence

Markers are weak signals. One instance of "delve" means almost nothing.

The honest thing a tool can do is point at the patterns that make writing read as machine‑generated — so a human can decide whether to change them.

The checker loads markers.json, runs lexical matching per category, applies regex patterns for structural and formatting tells, and computes burstiness plus type‑token ratio as perplexity proxies.