Show a model your old code and it writes your old bugs

Описание и применение

Модели ИИ воспроизводят ошибки из устаревшего кода, если в контекст включены старые компоненты. После миграции и добавления правила модели перестают писать баги, но могут ошибаться в интерфейсе, если не видят исходный код.

Задача

AI models tend to reproduce buggy patterns from legacy code when prompted with repository context.

Результат

Models replicated the exact bugs from legacy code when given the old repository context, but reused the shared component when the migration context or a single rule was provided. A single documentation paragraph was enough to steer the model away from hand‑rolling buggy code.

Рабочий процесс

• 1. Identify duplicated components and their bugs.
• 2. Refactor into a single shared component.
• 3. Add a test that fails if a new duplicate appears.
• 4. Run AI model experiments with different repository contexts.
• 5. Analyze model outputs for reuse vs. hand‑rolled code and bug reproduction.
• 6. Adjust prompts to include documentation or source to steer model.
• 7. Validate generated code against real prop list.
• 8. Iterate to improve model guidance.

Доказательства

Thirty‑two runs each, and not one crossed over. Same model, same instructions, 190 lines of hand‑written state machine in one case and a 41‑line wrapper in the other, decided entirely by what was lying around in the repository.

Источник

One paragraph of documentation, sitting next to nine hundred lines of exactly the code it forbids, flipped every single run.

Источник