Migration Diary: The Format Drift You Hit When Switching AI Providers (and the Test That Caught It)

Description and use

Migrating the code‑review bot to MonkeyCode’s free server required defining an API contract, building an adapter, and running regression tests to prevent JSON format drift.

Problem

Switching AI backends caused JSON format drift that broke downstream parsers, risking incorrect code‑review bot behavior.

Outcome

Successfully migrated to free backend with zero user‑visible errors; caught three format regressions before production use.

Workflow

• Extract contract from existing bot code
• Write adapter to map snake_case to camelCase, filter unexpected fields, default missing values
• Build regression test comparing old vs new outputs field‑by‑field
• Run shadow mode for 3 business days, logging differences to SQLite
• Use Python validator script on golden samples to catch regressions
• Switch to MonkeyCode once 95% match achieved

Evidence

The adapter was boring on purpose: it mapped snake_case keys to camelCase, filtered unexpected fields, and defaulted missing values.

Source

We ran the shadow mode for three business days, feeding every real pull request to both providers and logging the differences to a local SQLite table.

Source

It caught three format regressions before any user noticed, including one where the model returned a float instead of an integer for the confidence score.

Source