When AI Writes Both the API Integration and the Tests, What Are We Actually Verifying?

Description and use

Adding an independent verification layer (Kaktoos) ensures that API integrations generated by AI agents actually match the real API and OpenAPI specification, eliminating false positives.

Problem

AI agents that generate both API integration code and its tests may only confirm the agent’s own assumptions, leading to false confidence when the real API contract differs.

Outcome

Adding Kaktoos as a verification step uncovers mismatches between agent-generated assumptions and the real API, improving reliability of automated integrations.

Workflow

• 1. AI agent writes API integration code.
• 2. AI agent writes tests for the integration.
• 3. Integration code is executed against a mock or stub.
• 4. Kaktoos intercepts the integration call and validates the response against the OpenAPI spec and the real API.
• 5. Kaktoos reports discrepancies or confirms success.
• 6. Developers review Kaktoos results and adjust code or tests accordingly.

Evidence

AI agent → integration → Kaktoos → OpenAPI + real API → result

Source