48-Hour Field Notes: Unit Tests That Grew a Network

Description and use

Unit tests that inadvertently perform real HTTP requests increase CI runtime and cause flaky failures.

Problem

Unit tests unintentionally made live network calls, causing CI failures and hidden dependencies on external services.

Outcome

The experiment demonstrated that unit tests containing live network calls introduce hidden external dependencies, increase CI runtime, and cause flaky failures.

Workflow

• 1. Identify a fixture repo with a unit test that uses a mock.
• 2. Apply a diff that replaces the mock with a real httpx.get call.
• 3. Run the scanner script to detect network imports and calls in unit test files.
• 4. Execute the tests on a machine without VPN or local host overrides.
• 5. Observe CI time increase and test failure due to external dependency.

Evidence

CI time jumped from nine minutes to twenty-two, and one case failed with a TLS timeout against a hostname nobody on the team operates.

Source

The meter has to be mechanical: new network imports and call sites inside paths that look like unit tests, minus an allowlist for loopback.

Source