Open‑Sourced 410 AWS Agent Skills with Test Scores

Описание и применение

Открытый репозиторий 410 навыков AWS с прозрачными тестовыми оценками и проверками безопасности, позволяющий разработчикам доверять и улучшать автоматизацию.

Задача

Developers ship AWS agent skills that claim quality but lack transparent evaluation, risking unsafe or ineffective automation.

Результат

All 410 skills now have transparent, reproducible quality metrics and security checks, enabling developers to trust and improve agent automation.

Рабочий процесс

• 1. Write skill with SKILL.md and instruction files
• 2. Add test cases to skill folder
• 3. Run harness to execute tests via Nova Pro
• 4. Grade output with GPT‑OSS‑120B using softaworks skill‑judge rubric
• 5. Commit median scorecard JSON next to skill
• 6. Scan skill with OWASP Agentic Skills Top 10 detector
• 7. Generate skill.usf.yaml with risk tier, hash, and ed25519 signature
• 8. Publish skill to GitHub repo

Доказательства

Each skill carries its own test cases. A harness feeds them to Amazon Nova Pro, which does the task the way the skill instructs. Then gpt-oss-120b grades the output on the softaworks skill-judge rubric

Источник

First, every skill passed an independent implementation of the OWASP Agentic Skills Top 10 detectors: 410 skills, zero findings.

Источник