meta(skills): Add skill audit and safe metadata fixes
Add repo-wide auditing and targeted repair scripts for skill metadata. Fix truncated descriptions automatically, keep heading normalization conservative, and remove synthetic boilerplate sections that degrade editorial quality while regenerating repo indexes and catalogs. Fixes #365
This commit is contained in:
@@ -54,6 +54,15 @@ For pull requests that add or modify `SKILL.md`, GitHub also runs the automated
|
||||
- inline token/secret-style command examples,
|
||||
- deliberate allowlisted high-risk documentation commands via `<!-- security-allowlist: ... -->`.
|
||||
|
||||
### Additional Maintainer Audit
|
||||
|
||||
Use `npm run audit:skills` when you need a repo-wide report that goes beyond schema validation and answers:
|
||||
|
||||
- which skills are structurally valid but still need usability cleanup,
|
||||
- which skills still have truncated descriptions (issue `#365`),
|
||||
- which skills are missing examples or limitations,
|
||||
- and which skills have the highest concentration of warnings/errors.
|
||||
|
||||
---
|
||||
|
||||
## Support Levels
|
||||
@@ -74,6 +83,7 @@ The canonical validator is `tools/scripts/validate_skills.py`, but the recommend
|
||||
|
||||
```bash
|
||||
npm run validate
|
||||
npm run audit:skills
|
||||
npm run validate:references
|
||||
npm test
|
||||
npm run security:docs
|
||||
@@ -82,6 +92,7 @@ npm run security:docs
|
||||
Notes:
|
||||
|
||||
- `npm run validate` is the operational contributor gate.
|
||||
- `npm run audit:skills` is the maintainer-facing compliance/usability report for the full library.
|
||||
- `npm run security:docs` is required for command-heavy or risky skill content.
|
||||
- PRs that touch `SKILL.md` also get an automated `skill-review` GitHub Actions check.
|
||||
- `npm run validate:strict` is a useful hardening pass, but the repository still contains legacy skills that do not yet satisfy strict validation.
|
||||
|
||||
@@ -6,6 +6,7 @@ This document summarizes the repository coherence audit performed after the `app
|
||||
|
||||
- Conteggi e numeri (README, package.json, CATALOG)
|
||||
- Validazione skill (frontmatter, risk, "When to Use", link)
|
||||
- Audit repo-wide per skill (conformance + baseline usability)
|
||||
- Riferimenti incrociati (workflows.json, bundles.json, `docs/users/bundles.md`)
|
||||
- Documentazione (`docs/contributors/quality-bar.md`, `docs/contributors/skill-anatomy.md`, security/licenses)
|
||||
- Script e build (validate, index, readme, catalog, test)
|
||||
@@ -28,6 +29,16 @@ This document summarizes the repository coherence audit performed after the `app
|
||||
- requires explicit allowlists for deliberate command-delivery patterns,
|
||||
- and blocks token-like examples that look exploitable.
|
||||
|
||||
### 2b. Audit repo-wide per skill
|
||||
|
||||
- Added `tools/scripts/audit_skills.py` (also exposed as `npm run audit:skills`), which audits every `SKILL.md` and produces a per-skill status (`ok`, `warning`, `error`) with finding codes.
|
||||
- The audit is intentionally broader than `validate` and covers:
|
||||
- truncated descriptions that likely map to issue `#365`,
|
||||
- missing examples and missing limitations sections,
|
||||
- overly long `SKILL.md` files that should probably be split into `references/`,
|
||||
- plus the existing structural/safety checks (frontmatter, risk, `When to Use`, offensive disclaimer, dangling links).
|
||||
- Use `npm run audit:skills` for the maintainer view and `npm run audit:skills -- --json-out ... --markdown-out ...` when you want artifacts for triage or cleanup tracking.
|
||||
|
||||
### 3. Riferimenti incrociati
|
||||
|
||||
- Added `tools/scripts/validate_references.py` (also exposed as `npm run validate:references`), which verifies:
|
||||
@@ -61,6 +72,7 @@ This document summarizes the repository coherence audit performed after the `app
|
||||
```bash
|
||||
npm run validate # validazione skill (soft)
|
||||
npm run validate:strict # hardening / diagnostic pass
|
||||
npm run audit:skills # audit completo per skill con finding codes e status
|
||||
npm run validate:references # workflow, bundle, and docs/users/bundles.md references
|
||||
npm run security:docs # documentation command-risk scan (required for security-sensitive guidance)
|
||||
npm run build # chain + catalog
|
||||
|
||||
Reference in New Issue
Block a user