I labeled 558 AGENTS.md files. Here’s what they say — and what almost nobody writes down

Description and use

The project collected 558 AGENTS.md files, labeled them with a 9‑category rule‑based taxonomy, and validated the classifier with 92% precision/70% recall on English and 88%/73% on Chinese. The resulting dataset and agent‑charters tool enable developers to audit their own AGENTS.md files against a reproducible baseline.

Problem

Developers lack a systematic view of AGENTS.md content, making it hard to ensure completeness and consistency across repos.

Outcome

The project produced a reproducible baseline of AGENTS.md content, revealing that prohibitions and build/test commands dominate, while gotchas are rare. The tool enables developers to audit their own files against this baseline.

Workflow

• Collect AGENTS.md files from public repos
• Apply rule‑based classifier to assign 9 categories
• Validate classifier against blind‑labeled held‑out samples
• Publish dataset and tooling (agent‑charters)
• Invite external users to run compare and provide feedback

Evidence

92% precision / 70% recall on 55 English files, 88% / 73% on 50 Chinese files

Source

49% point to some other file; 15% route to a knowledge or rules directory

Source