What happens when enterprise requirements hit Strands, LangGraph, and CrewAI - 45 runs measured

Description and use

Three agent frameworks were tested on a news‑digest task requiring audit trails and human approval. Strands sometimes outputs empty results, LangGraph provides full audit visibility, and CrewAI is simple but can loop on rejection.

Problem

Enterprise AI systems must satisfy audit trails and human approval gates, but agent frameworks differ in how they enforce these constraints.

Outcome

Strands produced empty outputs in 3 runs; LangGraph and CrewAI never did. LangGraph had 0% tool order visibility in traces, CrewAI had 50% tool order and args visibility, while Strands had 100% for all audit facts but 2 silent failures. All frameworks achieved 100% structured JSON compliance.

Workflow

• 1. Define a news‑digest task with a simulated destructive publish action.
• 2. Implement human approval gates in each framework (LangGraph interrupt, CrewAI Task(human_input=True), Strands prompt‑only).
• 3. Run 18 experiments to test approval enforcement.
• 4. Record all runs via a shared recorder proxy for trace analysis.
• 5. Analyze audit‑trail completeness across 36 runs.
• 6. Test structured JSON output compliance in 9 runs.
• 7. Aggregate results across 45 total runs.
• 8. Identify failure modes and audit gaps.
• 9. Publish findings and provide reproducible code.

Evidence

Strands finished with an empty output three times while exiting

Source

LangGraph suspended 6/6 runs and resumed in 0.0s

Source

Strands is model‑driven, so everything the model saw and reasoned about stays in the trace

Source