8 Frameworks in the Demo, 3 Rows in the Conformance Table: I Ran Microsoft's agent-hooks CTK

Description and use

Testing Microsoft agent‑hooks against various frameworks revealed that only three of the eight claimed supported; the rest lack adapters. The CTK runs quickly but skips four vectors due to missing incremental_output, indicating incomplete streaming tests.

Problem

Determining which frameworks are truly supported by Microsoft’s agent‑hooks conformance kit and understanding gaps between demo claims and actual adapter availability.

Outcome

The CTK runs in 0.37 s, producing 73 vectors with 69 passes, 4 skips, and 0 failures. Skipped vectors are due to missing incremental_output capability, indicating untested streaming behavior. Only three frameworks have claim entries; six lack adapters, making them unsupported in the current SDK.

Workflow

• 1. Install agent‑hooks SDK (e.g., agent-hooks-sdk==0.1.0b2).
• 2. Import CTK functions: from agent_hooks.ctk import load_vectors, run_vectors and ReferenceHarness from agent_hooks.ctk.reference.
• 3. Load vectors: vectors = load_vectors().
• 4. Run vectors: results = run_vectors(ReferenceHarness, vectors).
• 5. Inspect results: print(len(results), sum(r.status == "pass" for r in results)).
• 6. Review skipped vectors and reason field for capability gaps.
• 7. Compare results against the claim table in conformance/CLAIMS.md.
• 8. Verify adapter presence by grepping for framework names in the agent_hooks package.
• 9. If adapter missing, note that the framework is not yet supported.
• 10. Use findings to decide whether to adopt or request adapter integration.

Evidence

"produces the identical 20-row decision stream"

Source

On agent-hooks-sdk==0.1.0b2, Python 3.13, that's 73 vectors, 69 pass, 4 skip, 0 fail — in 0.37 seconds.

Source

The four skips are the interesting part, because they're not failures: ... missing capabilities: ["incremental_output"]

Source

Three rows. Six of the eight frameworks named in the launch post — LangGraph, the OpenAI Agents SDK, Semantic Kernel, LlamaIndex, CrewAI, the Claude Agent SDK — have nothing filed.

Source

If you want agent-hooks inside a LangGraph or OpenAI Agents SDK loop today, you write the adapter, because the four-method harness interface is the interface and nobody has written yours yet.

Source