2,300 Passing Tests and a CSV That Loaded One Row

Description and use

Connector tests must involve real data, not just mocks, to ensure data actually moves.

Problem

A data platform’s connector tests passed but did not actually load data, revealing that passing tests do not guarantee functional data movement.

Outcome

After auditing, Datanika now requires real data movement to be verified in tests, preventing silent failures and improving connector reliability.

Workflow

• 1. Identify connectors with only mock‑based tests.
• 2. Add real‑world probes that spin up containers and load actual rows.
• 3. Ensure tests fail against broken code before the fix is merged.
• 4. Pin known‑broken connectors red instead of skipping them.
• 5. Verify that a row appears in the destination after a successful run.

Evidence

The mock happily played the part of a working connector, because a mock will play any part you write for it.

Source

A connector is not done when its tests pass. It is done when a row has been observed in the destination.

Source