Show a model your old code and it writes your old bugs

说明与用途

AI 模型在提示旧代码时会复制旧错误。迁移后添加规则可阻止模型手工编写错误代码，但若缺少源文件，可能会虚构属性。

问题

AI models tend to reproduce buggy patterns from legacy code when prompted with repository context.

结果

Models replicated the exact bugs from legacy code when given the old repository context, but reused the shared component when the migration context or a single rule was provided. A single documentation paragraph was enough to steer the model away from hand‑rolling buggy code.

工作流

• 1. Identify duplicated components and their bugs.
• 2. Refactor into a single shared component.
• 3. Add a test that fails if a new duplicate appears.
• 4. Run AI model experiments with different repository contexts.
• 5. Analyze model outputs for reuse vs. hand‑rolled code and bug reproduction.
• 6. Adjust prompts to include documentation or source to steer model.
• 7. Validate generated code against real prop list.
• 8. Iterate to improve model guidance.

证据

Thirty‑two runs each, and not one crossed over. Same model, same instructions, 190 lines of hand‑written state machine in one case and a 41‑line wrapper in the other, decided entirely by what was lying around in the repository.

来源

One paragraph of documentation, sitting next to nine hundred lines of exactly the code it forbids, flipped every single run.

来源