A Free‑Tier Trace Loop for Agent Run Regression

说明与用途

使用 MonkeyCode 免费资源的自动化跟踪比较循环，帮助检测代理运行中的回归问题，无需额外费用。

问题

Agent runs are stochastic; fixing one bug often reveals another. Detecting regressions requires comparing full trace logs, which traditionally needs paid server and model resources.

结果

Implemented a zero‑cost trace comparison loop that automatically detects regressions in agent behavior using free MonkeyCode resources.

工作流

• 1. Configure MonkeyCode API key and base URL via environment variables.
• 2. Run FastAPI service that stores each agent run’s events in SQLite.
• 3. After an agent run finishes, POST the list of tool calls, inputs, and outputs to /traces endpoint.
• 4. To compare two runs, send a GET request to /compare?run1=<run_id1>&run2=<run_id2>.
• 5. The service calls MonkeyCode’s model to produce a bullet‑point diff and verdict.
• 6. Review the diff to identify behavioral changes such as wrong tool usage or argument order.

证据

MonkeyCode is an open‑source project that offers two things relevant to this workflow: free model access and a free server option.

来源

The loop I use is simple: record every tool call and input/output pair from an agent run into a structured store; serialize each run into a single document; then ask a model to diff two documents and highlight behavioral changes.

来源