Migration Diary: The Format Drift You Hit When Switching AI Providers (and the Test That Caught It)

Описание и применение

Перенос бота проверки кода на бесплатный сервер MonkeyCode потребовал создания контракта API, адаптера и регрессионных тестов, чтобы избежать сдвига формата JSON.

Задача

Switching AI backends caused JSON format drift that broke downstream parsers, risking incorrect code‑review bot behavior.

Результат

Successfully migrated to free backend with zero user‑visible errors; caught three format regressions before production use.

Рабочий процесс

• Extract contract from existing bot code
• Write adapter to map snake_case to camelCase, filter unexpected fields, default missing values
• Build regression test comparing old vs new outputs field‑by‑field
• Run shadow mode for 3 business days, logging differences to SQLite
• Use Python validator script on golden samples to catch regressions
• Switch to MonkeyCode once 95% match achieved

Доказательства

The adapter was boring on purpose: it mapped snake_case keys to camelCase, filtered unexpected fields, and defaulted missing values.

Источник

We ran the shadow mode for three business days, feeding every real pull request to both providers and logging the differences to a local SQLite table.

Источник

It caught three format regressions before any user noticed, including one where the model returned a float instead of an integer for the confidence score.

Источник