Smithery is now a part of Arcade.dev! Read more in our announcement here
Iteratively improve skill frontmatter compliance using the Ralph loop pattern...
When user says "sensei help" or asks how to use sensei, show this:
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
ā SENSEI - Skill Frontmatter Compliance Improver ā
ā āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā£
ā ā
ā USAGE: ā
ā Run sensei on <skill-name> # Single skill ā
ā Run sensei on <skill1>, <skill2>, ... # Multiple skills ā
ā Run sensei on all skills # All skills ā
ā ā
ā EXAMPLES: ā
ā Run sensei on appinsights-instrumentation ā
ā Run sensei on azure-ai, azure-compute ā
ā ā
ā WHAT IT DOES: ā
ā 1. READ - Load skill's SKILL.md and tests ā
ā 2. VERIFY - Compare skill frontmatter with convention ā
ā 3. SCAFFOLD - Create tests from frontmatter if missing ā
ā 4. IMPROVE - Add WHEN: triggers ā
ā 5. TEST - Run tests, fix if needed ā
ā 6. SUMMARY - Show before/after with suggestions ā
ā 7. PROMPT - Ask: Commit, Create Issue, or Skip? ā
ā 8. REPEAT - Until routing tests pass ā
āāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāāā
For each skill, execute this loop until the frontmatter aligns with convention, have thorough routing tests AND routing tests pass:
plugins/{plugin}/skills/{skill-name}/SKILL.md, and vally eval suites in evals/{skill-name}/*.yaml.evals/{skill-name}/ doesn't exist, follow instructions in vally-eval skill to scaffold a set of routing tests. The routing tests test if the skill can be invoked for target user prompts. Generate user prompts that match the target scenario of the skill's description.vally-eval skill to run the routing tests. If there are failed tests, suggest fixes.plugins/{plugin}/skills/ - these are the Azure skills used by Copilot.github/skills/ should be left as is