I built Agent Review Studio: a local-first workbench for agent harness evaluations

说明与用途

工程师现在拥有一个工具，可以深入分析 AI 代理的运行，查看声明、来源和执行跟踪，标记失败并在不进行自动微调的情况下改进模型。

问题

Engineers lacked a comprehensive way to inspect AI agent runs beyond final answers, making it hard to trace claims, evidence, and execution details.

结果

Engineers can now systematically review agent runs, identify failures, generate golden evaluation cases, and refine prompts, tools, and memory policies, improving agent quality without automatic fine-tuning.

工作流

• Create a workspace for the agent or harness
• Run or import a versioned evaluation
• Inspect claims alongside source evidence and execution traces
• Label failures and write corrections
• Score the run across five quality dimensions
• Preserve immutable review history
• Compare baseline and candidate runs
• Export reviewed improvement data

证据

Agent Review Studio lets an engineer: Create a workspace for any agent or harness. Run or import a versioned evaluation. Inspect claims beside their source evidence and execution traces. Label failures and write the correction. Score the complete run across five quality dimensions. Preserve immutable review history. Compare baseline and candidate runs. Export reviewed improvement data.

来源