From SkillCheck to TraceMantle: Tracking Agent Skill Changes and Test Evidence

Описание и применение

TraceMantle расширяет SkillCheck, отслеживая все файлы навыка, связывая результаты тестов с изменениями пакета и автоматически определяя, какие проверки нужно повторить.

Задача

Developers needed a tool that not only validates AI agent skill files but also tracks changes in supporting resources and determines whether existing test evidence remains applicable.

Результат

TraceMantle extends SkillCheck by tracking all skill package files, linking test evidence to package changes, and automating evidence comparison, enabling developers to quickly identify which tests need rerunning and ensuring evidence remains trustworthy.

Рабочий процесс

• 1. Run TraceMantle on the skill package to validate frontmatter, references, size limits, and compatibility.
• 2. Generate a bundle manifest that fingerprints all files in the package.
• 3. Import test evidence from a Promptfoo export (version‑pinned).
• 4. Compare the imported evidence against the current package and a trusted policy to identify changed inputs, incompatible results, and checks requiring rerun.
• 5. Review TraceMantle output (text, JSON, or GitHub annotations) and decide whether to approve evidence or rerun tests.

Доказательства

TraceMantle retains that functionality and adds support for tracking the files in a skill package, comparing changes, and checking whether previous test evidence still applies.

Источник

The expanded functionality adds three related capabilities: Bundle manifests, Evidence imports, Evidence comparison.

Источник