What happens when enterprise requirements hit Strands, LangGraph, and CrewAI - 45 runs measured
Описание и применение
Три агентских фреймворка (Strands, LangGraph, CrewAI) протестированы на задаче создания новостного дайджеста с требованиями к аудиту и одобрению. Strands иногда выдаёт пустой результат, LangGraph обеспечивает полный аудит, CrewAI прост в использовании, но может зациклиться при отказе.
Задача
Enterprise AI systems must satisfy audit trails and human approval gates, but agent frameworks differ in how they enforce these constraints.
Результат
Strands produced empty outputs in 3 runs; LangGraph and CrewAI never did. LangGraph had 0% tool order visibility in traces, CrewAI had 50% tool order and args visibility, while Strands had 100% for all audit facts but 2 silent failures. All frameworks achieved 100% structured JSON compliance.
Рабочий процесс
- 1. Define a news‑digest task with a simulated destructive publish action.
- 2. Implement human approval gates in each framework (LangGraph interrupt, CrewAI Task(human_input=True), Strands prompt‑only).
- 3. Run 18 experiments to test approval enforcement.
- 4. Record all runs via a shared recorder proxy for trace analysis.
- 5. Analyze audit‑trail completeness across 36 runs.
- 6. Test structured JSON output compliance in 9 runs.
- 7. Aggregate results across 45 total runs.
- 8. Identify failure modes and audit gaps.
- 9. Publish findings and provide reproducible code.
Доказательства
Strands finished with an empty output three times while exiting
LangGraph suspended 6/6 runs and resumed in 0.0s
Strands is model‑driven, so everything the model saw and reasoned about stays in the trace
- Дата записи: 2026-09-22T13:15:36.480Z