research-ledger: a skill that helps one person and several AI models not fool themselves
说明与用途
"research‑ledger" 是一套规则和工具,帮助研究人员系统地评估想法,将隐喻转化为可测量的测试,记录先前研究,预注册实验,区分试点和确认性测试,使用不同模型进行盲测,并维护不可变的结果日志,从而降低自我欺骗风险。
问题
Researchers risk fooling themselves when relying on AI models, leading to false confidence in ideas, mechanisms, thresholds, and novelty claims.
结果
The skill has led to most ideas being classified as DEAD or LANDED, demonstrating effective filtering. It also produced a clean kill of a selective learning rule, and cross‑lab reviews improved review quality. The method has not produced new discoveries but has documented failures and reduced self‑deception.
工作流
- Sieve ideas within 5 minutes against a short checklist
- Translate metaphors into measurable tests or precise statements
- Record prior art searches with date, terms, and nearest source
- Preregister predictions, thresholds, controls, and statistical power before running experiments
- Differentiate pilot runs from confirmatory tests
- Conduct blind rounds at milestones using seeds shared across models from different labs
- Assign one of five verdicts (DEAD, LANDED, BRIDGE CANDIDATE, BRIDGE, EXPLANATION)
- Maintain an append‑only ledger of all outcomes and lessons learned
- Add new terms to a glossary with standard names or note lack of standard term
证据
The main risk was never a lack of ideas, it was fooling myself.
Sieve first. Most ideas die in five minutes against a short checklist.
Preregistration. Predictions, thresholds, controls and statistical power are frozen before the run.
Blind rounds. At milestones, the same seed goes to two or three models from different labs, in clean conversations, with no project context.
The ledger is mostly DEAD and LANDED entries.
- 记录日期: 2026-10-13T07:41:24.006Z