Your README Is Lying to You: Test Your Onboarding Docs Like You Test Code

Description and use

Automated README testing reduces onboarding friction by detecting missing dependencies, version drift, and omitted steps before contributors encounter them.

Problem

README files often become outdated or incomplete, causing new contributors to fail at early setup steps.

Outcome

The approach uncovers hidden dependencies, version drift, and missing steps before contributors encounter them, reducing onboarding friction and keeping the README reliable.

Workflow

• 1. Invite a new team member or intern who has never seen the repo to read the README and think aloud while following the instructions.
• 2. Record friction points in a friction‑log.md table, noting blockers and minor issues.
• 3. Tag executable code blocks in the README with a custom language identifier (e.g., ```bash readme-test).
• 4. Run a Python script that extracts these blocks and executes them in a single shell session with set -euxo pipefail.
• 5. Execute the script inside a minimal Docker container that only installs the prerequisites listed in the README.
• 6. Add a GitHub Actions workflow that triggers on changes to README.md, docs/**, package.json, or pyproject.toml and runs the readme‑test script weekly.
• 7. Review CI failures, fix the README, and iterate.

Evidence

"Bạn chỉ được đọc README, không được hỏi mình."

Source

"đánh dấu những code block cần test bằng một tag riêng trong info string của fence, ví dụ bash readme-test."

Source

"docker run --rm -v \"$PWD\":/src:ro ubuntu:24.04 bash -c 'apt-get update -qq ... python3 scripts/readme_runner.py README.md'"

Source

"trigger khi README đổi và chạy định kỳ hàng tuần"

Source