Open‑Sourced 410 AWS Agent Skills with Test Scores

Description and use

Open‑source library of 410 AWS agent skills with transparent test scores and security checks, enabling developers to trust and refine automation.

Problem

Developers ship AWS agent skills that claim quality but lack transparent evaluation, risking unsafe or ineffective automation.

Outcome

All 410 skills now have transparent, reproducible quality metrics and security checks, enabling developers to trust and improve agent automation.

Workflow

• 1. Write skill with SKILL.md and instruction files
• 2. Add test cases to skill folder
• 3. Run harness to execute tests via Nova Pro
• 4. Grade output with GPT‑OSS‑120B using softaworks skill‑judge rubric
• 5. Commit median scorecard JSON next to skill
• 6. Scan skill with OWASP Agentic Skills Top 10 detector
• 7. Generate skill.usf.yaml with risk tier, hash, and ed25519 signature
• 8. Publish skill to GitHub repo

Evidence

Each skill carries its own test cases. A harness feeds them to Amazon Nova Pro, which does the task the way the skill instructs. Then gpt-oss-120b grades the output on the softaworks skill-judge rubric

Source

First, every skill passed an independent implementation of the OWASP Agentic Skills Top 10 detectors: 410 skills, zero findings.

Source