Stop Dumping Raw Pytest Logs Into Your Coding Agent: How a Structured JSON Test Filter Cuts Input Tokens by 94.1%

Описание и применение

Фильтрация вывода тестов в структурированный JSON уменьшила токены на 94 % и сократила стоимость API на 87 %.

Задача

Autonomous coding agents consume excessive tokens when raw CLI test runner output is appended to the conversation history, inflating input size and cost.

Результат

Using a structured JSON test filter (TokenKillerFilterRunner) compressed raw CLI output by 98.6 % and cut cumulative input tokens by 94.1 %, enabling the agent to fix all failures in a single edit turn and reducing API cost by 87 %.

Рабочий процесс

• Run pytest with verbose output
• Append raw stdout to conversation history
• Agent attempts partial patch, triggering second test run
• Append second raw stdout
• Agent submits full patch

Доказательства

When an agent executes pytest -vv --tb=long, cargo test, or npm test on a realistic repository where 76 tests pass and 4 fail, dumping the raw terminal output (ANSI color codes, environment banners, 76 [PASS] lines, captured DEBUG stdout, and multi-frame internal tracebacks) directly into chat history adds 60,000+ characters (28,000+ tokens) per test run.

Источник

Mode B: Wraps the test runner in a deterministic preprocessor that strips ANSI escape codes, environment banners, all 76 passing tests, and internal framework frames—returning only a 1,506-character structured JSON summary of the 4 failed assertions.

Источник

Compressing raw CLI test output from 117,391 chars ➔ 1,604 chars (-98.6% reduction, 73.2x compression) enabled Mode B to fix all three modules in a single edit turn (2 LLM calls total vs. 3 in Mode A).

Источник

Mode B cut cumulative input tokens by -94.1% (92,385 ➔ 5,445 tokens, 16.97x efficiency)

Источник