We open-sourced the tool that failed its own kill rule

Description and use

<p>Before we ran the experiment, we wrote down the rule that would kill it.</p> <p>The product under test was a pipeline that builds skill files for AI agents: those structured instruction documents, SKILL.md and friends, that tell an agent when to trigger, what to do, and what never to do. Our hypothesis was the obvious one. Skills built through a structured pipeline, with a requirements interview, controlled generation, and a mechanical audit, should beat skills the model improvises for itself in one shot. We pre-registered four predictions and one kill criterion: if the pipeline could not m

Problem

<p>Before we ran the experiment, we wrote down the rule that would kill it.</p> <p>The product under test was a pipeline that builds skill files for AI agents: those structured instruction documents, SKILL.md and friends, that tell an agent when to trigger, what to do, and what never to do. Our hypothesis was the obvious one. Skills built through a structured pipeline, with a requirements interview, controlled generation, and a mechanical audit, should beat skills the model improvises for itself in one shot. We pre-registered four predictions and one kill criterion: if the pipeline could not m

Outcome

<p>Before we ran the experiment, we wrote down the rule that would kill it.</p> <p>The product under test was a pipeline that builds skill files for AI agents: those structured instruction documents, SKILL.md and friends, that tell an agent when to trigger, what to do, and what never to do. Our hypothesis was the obvious one. Skills built through a structured pipeline, with a requirements interview, controlled generation, and a mechanical audit, should beat skills the model improvises for itself in one shot. We pre-registered four predictions and one kill criterion: if the pipeline could not m

Evidence

We open-sourced the tool that failed its own kill rule

Source