I Read a 4-Million-Line Study of AI Coding Agents. It Described My Own Tool.

Описание и применение

Исследование 4 млн строк кода раскрывает, что продуктом является не модель, а «харнесс»; большинство систем используют собственные циклы и избегают общих фреймворков. Автор проверил настройки сжатия и хуки в Pi, подтвердив выводы работы.

Задача

Developers need to understand how coding‑agent harnesses differ from the underlying LLMs and how design choices affect safety, performance, and extensibility.

Результат

The study demonstrates that harness design, not just the LLM, drives agent behavior. Key findings include the absence of general frameworks, lack of embeddings for code retrieval, and that loop sophistication does not predict performance. The author verified Pi’s compaction logic, hooks, and safety stance, confirming the study’s claims.

Рабочий процесс

• 1. Read the study and identify key observations. 2. Map observations to the author’s Pi harness. 3. Verify claims against Pi’s documentation (e.g., compaction settings, hooks, safety design). 4. Apply the study’s 18 recommendations to future harness designs.
• 5. Monitor convergence of design patterns across the ecosystem (e.g., skill formats, ACP usage).

Доказательства

Agent = Model + Harness. The harness is everything except the model.

Источник

Zero import a general‑purpose agentic framework. Not LangChain, not LangGraph, not AutoGen, not CrewAI, not LlamaIndex, not Pydantic AI, not Semantic Kernel.

Источник

Pi compacts at contextWindow − 16,384, keeping 20K recent, and merges summaries iteratively rather than re‑summarizing from scratch.

Источник

The paper’s conclusion: architectural complexity does not predict how well an agent scores, but it does predict production readiness.

Источник