From SkillCheck to TraceMantle: Tracking Agent Skill Changes and Test Evidence

说明与用途

TraceMantle 扩展了 SkillCheck，跟踪技能包中的所有文件，将测试证据与包更改关联，并自动比较证据，帮助开发者快速识别需要重新运行的测试。

问题

Developers needed a tool that not only validates AI agent skill files but also tracks changes in supporting resources and determines whether existing test evidence remains applicable.

结果

TraceMantle extends SkillCheck by tracking all skill package files, linking test evidence to package changes, and automating evidence comparison, enabling developers to quickly identify which tests need rerunning and ensuring evidence remains trustworthy.

工作流

• 1. Run TraceMantle on the skill package to validate frontmatter, references, size limits, and compatibility.
• 2. Generate a bundle manifest that fingerprints all files in the package.
• 3. Import test evidence from a Promptfoo export (version‑pinned).
• 4. Compare the imported evidence against the current package and a trusted policy to identify changed inputs, incompatible results, and checks requiring rerun.
• 5. Review TraceMantle output (text, JSON, or GitHub annotations) and decide whether to approve evidence or rerun tests.

证据

TraceMantle retains that functionality and adds support for tracking the files in a skill package, comparing changes, and checking whether previous test evidence still applies.

来源

The expanded functionality adds three related capabilities: Bundle manifests, Evidence imports, Evidence comparison.

来源