fix: streamline pr and release workflow (#289)

Co-authored-by: sck_0 <samujackson1337@gmail.com>
2026-03-13 14:20:49 +01:00
parent 5655f9b0a8
commit e325b0ee30
17 changed files with 1100 additions and 172 deletions
--- a/.github/MAINTENANCE.md
+++ b/.github/MAINTENANCE.md
@@ -35,7 +35,11 @@ If you touch **any of these**:
 - Running `npm run chain` is **NOT optional**.
 - Running `npm run catalog` is **NOT optional**.

-For contributor PRs, generated drift is now **informational** in CI because shared registry artifacts are auto-synced on `main` after merge. Contributors should still run the chain locally so the PR content is reviewable and maintainers can reproduce the generated output when needed.
+For contributor PRs, the contract is now **source-only**:
+
+- contributors should not commit `CATALOG.md`, `skills_index.json`, or `data/*.json`
+- PR CI previews generated drift but does not require those files in the branch
+- `main` remains the only canonical owner of derived registry artifacts

 If `main` CI fails with:

@@ -78,11 +82,11 @@ Before ANY commit that adds/modifies skills, run the chain:

 3.  **COMMIT GENERATED FILES**:
    ```bash
-    git add README.md skills_index.json data/catalog.json data/bundles.json data/aliases.json CATALOG.md
+    git add README.md skills_index.json data/skills_index.json data/catalog.json data/bundles.json data/aliases.json CATALOG.md
    git commit -m "chore: sync generated files"
    ```
    > 🔴 **CRITICAL for direct `main` work**: If you skip this on maintainer work that lands directly on `main`, CI will fail with "Detected uncommitted changes".
-    > For contributor PRs, generated drift is allowed in CI and is auto-synced after merge.
+    > For contributor PRs, do **not** include derived registry artifacts. CI blocks direct edits to those files and previews drift separately.
    > See [`docs/maintainers/ci-drift-fix.md`](../docs/maintainers/ci-drift-fix.md) for details.

 ### B. When You Merge a PR (Step-by-Step)
@@ -92,14 +96,14 @@ Before ANY commit that adds/modifies skills, run the chain:
 **Before merging:**

 1.  **CI is green** — Validation, reference checks, tests, and generated artifact steps passed (see [`.github/workflows/ci.yml`](workflows/ci.yml)).
-2.  **Generated drift understood** — On pull requests, generator drift is informational only. Do not block a good PR solely because `README.md`, `CATALOG.md`, or catalog/index files would be regenerated. `main` auto-syncs those artifacts after merge.
+2.  **Generated drift understood** — On pull requests, generator drift is informational only. Do not block a good PR solely because canonical artifacts would be regenerated. Also do not accept PRs that directly edit `CATALOG.md`, `skills_index.json`, or `data/*.json`; those files are `main`-owned.
 3.  **Quality Bar** — PR description confirms the [Quality Bar Checklist](.github/PULL_REQUEST_TEMPLATE.md) (metadata, risk label, credits if applicable).
 4.  **Issue link** — If the PR fixes an issue, the PR description should contain `Closes #N` or `Fixes #N` so GitHub auto-closes the issue on merge.

 **How you merge:**

 - **Always merge via GitHub** so the PR shows as **Merged** and the contributor gets credit. Use **"Squash and merge"**. Do **not** integrate locally and then close the PR — that would show "Closed" and the contributor would not get proper attribution.
- **If the PR has merge conflicts:** Resolve them **on the PR branch** (you or the contributor: merge `main` into the PR branch, fix conflicts, run `npm run chain` and `npm run catalog` if needed, push). For generated registry files, prefer keeping `main`'s side and regenerating rather than hand-editing conflicts. Then use **"Squash and merge"** on GitHub. Full steps: [docs/maintainers/merging-prs.md](../docs/maintainers/merging-prs.md).
+- **If the PR has merge conflicts:** Resolve them **on the PR branch** (you or the contributor: merge `main` into the PR branch, fix conflicts, drop derived registry files from the branch if they appear, push). For generated registry files, prefer keeping `main`'s side rather than hand-editing conflicts. Then use **"Squash and merge"** on GitHub. Full steps: [docs/maintainers/merging-prs.md](../docs/maintainers/merging-prs.md).
 - **Rare exception:** Only if merging via GitHub is not possible, you may integrate locally and close the PR; in that case you **must** add a Co-authored-by line to the commit and explain in a comment. Prefer to avoid this so PRs are always **Merged**.

 **If a PR was closed after local integration (reopen and merge):**
@@ -116,8 +120,8 @@ If a PR was integrated via local squash and then **closed** (so it shows "Closed
    ```bash
    git merge origin/main -m "chore: merge main to resolve conflicts"
    ```
-    For conflicts in generated/registry files (`README.md`, `CATALOG.md`, `data/catalog.json`, etc.), keep **main's version**:  
-    `git checkout --theirs README.md CATALOG.md data/catalog.json` (and any other conflicted files), then `git add` them.
+    For conflicts in generated/registry files (`CATALOG.md`, `data/catalog.json`, etc.), keep **main's version** and remove those derived files from the PR branch:
+    `git checkout --theirs CATALOG.md data/catalog.json` (and any other derived files), then `git add` them.
 4.  **Commit the merge** (if not already done):  
    `git commit -m "chore: merge main to resolve conflicts" --no-edit`
 5.  **Push to the contributor's fork.** Add their fork as a remote if needed (replace `USER` and `BRANCH` with the PR head owner and branch from the PR page):
@@ -250,27 +254,24 @@ Reject any PR that fails this:
 When cutting a new version, follow the maintainer playbook in [`docs/maintainers/release-process.md`](../docs/maintainers/release-process.md).

 **Release checklist (order matters):**  
-Operational verification → Changelog → Bump `package.json` (and README if needed) → Commit & push → Create GitHub Release with tag matching `package.json` → npm publish (manual or via CI) → Close remaining linked issues.
+Preflight verification → Changelog → `npm run release:prepare -- X.Y.Z` → `npm run release:publish -- X.Y.Z` → npm publish (manual or via CI) → Close remaining linked issues.

 ---

 1.  **Run release verification**:
    ```bash
-    npm run validate
-    npm run validate:references
-    npm run sync:all
-    npm run test
-    npm run app:build
+    npm run release:preflight
    ```
    Optional diagnostic pass:
    ```bash
    npm run validate:strict
    ```
 2.  **Update Changelog**: Add the new release section to `CHANGELOG.md`.
-3.  **Bump Version**:
-    - Update `package.json` → `"version": "X.Y.Z"` (source of truth for npm).
-    - Update version header in `README.md` if it displays the number.
-    - One-liner: `npm version patch` (or `minor`/`major`) — bumps `package.json` and creates a git tag; then amend if you need to tag after release.
+3.  **Prepare commit and tag locally**:
+    ```bash
+    npm run release:prepare -- X.Y.Z
+    ```
+    This validates the release, aligns versioned files, writes the release notes artifact, creates the release commit, and creates the local tag.
 4.  **Create GitHub Release** (REQUIRED):

    > ⚠️ **CRITICAL**: Pushing a tag (`git push --tags`) is NOT enough. You must create a **GitHub Release Object** for it to appear in the sidebar and trigger the NPM publish workflow.
@@ -278,9 +279,7 @@ Operational verification → Changelog → Bump `package.json` (and README if ne
    Use the GitHub CLI:

    ```bash
-    # Prepare release notes (copy the new section from CHANGELOG.md into docs/maintainers/release-process.md, or use CHANGELOG excerpt)
-    # Then create the tag AND the release page (tag must match package.json version, e.g. v4.1.0)
-    gh release create v4.0.0 --title "v4.0.0 - [Theme Name]" --notes-file docs/maintainers/release-process.md
+    npm run release:publish -- X.Y.Z
    ```

    **Important:** The release tag must match `package.json`'s version. The [Publish to npm](workflows/publish-npm.yml) workflow runs on **Release published** and will run `npm publish`; npm rejects republishing the same version.
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -2,6 +2,18 @@

 Please include a summary of the change and which skill is added or fixed.

+## Change Classification
+
+- [ ] Skill PR
+- [ ] Docs PR
+- [ ] Infra PR
+
+## Issue Link (Optional)
+
+Use this only when the PR should auto-close an issue:
+
+`Closes #N` or `Fixes #N`
+
 ## Quality Bar Checklist ✅

 **All items must be checked before merging.**
@@ -12,13 +24,9 @@ Please include a summary of the change and which skill is added or fixed.
 - [ ] **Triggers**: The "When to use" section is clear and specific.
 - [ ] **Security**: If this is an _offensive_ skill, I included the "Authorized Use Only" disclaimer.
 - [ ] **Local Test**: I have verified the skill works locally.
- [ ] **Repo Checks**: I ran `npm run validate:references` if my change affected docs, bundles, workflows, or generated artifacts.
+- [ ] **Repo Checks**: I ran `npm run validate:references` if my change affected docs, workflows, or infrastructure.
+- [ ] **Source-Only PR**: I did not manually include generated registry artifacts (`CATALOG.md`, `skills_index.json`, `data/*.json`) in this PR.
 - [ ] **Credits**: I have added the source credit in `README.md` (if applicable).
-
-## Type of Change
-
- [ ] New Skill (Feature)
- [ ] Documentation Update
- [ ] Infrastructure
+- [ ] **Maintainer Edits**: I enabled **Allow edits from maintainers** on the PR.

 ## Screenshots (if applicable)
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -1,8 +1,5 @@
 name: Skills Registry CI

-permissions:
-  contents: write
-
 on:
  push:
    branches: ["main"]
@@ -10,9 +7,65 @@ on:
    branches: ["main"]
  workflow_dispatch:

+permissions:
+  contents: read
+
 jobs:
-  validate-and-build:
+  pr-policy:
+    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
+    outputs:
+      primary_category: ${{ steps.intake.outputs.primary_category }}
+      categories: ${{ steps.intake.outputs.categories }}
+      requires_references: ${{ steps.intake.outputs.requires_references }}
+      direct_derived_changes_count: ${{ steps.intake.outputs.direct_derived_changes_count }}
+      has_quality_checklist: ${{ steps.intake.outputs.has_quality_checklist }}
+      has_issue_link: ${{ steps.intake.outputs.has_issue_link }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Set up Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: "lts/*"
+
+      - name: Fetch base branch
+        run: git fetch origin "${{ github.base_ref }}"
+
+      - name: Intake PR change
+        id: intake
+        run: |
+          node tools/scripts/pr_preflight.js \
+            --base "origin/${{ github.base_ref }}" \
+            --head "HEAD" \
+            --event-path "$GITHUB_EVENT_PATH" \
+            --no-run \
+            --write-github-output \
+            --write-step-summary
+
+      - name: Enforce PR source-only contract
+        run: |
+          if [ "${{ steps.intake.outputs.direct_derived_changes_count }}" != "0" ]; then
+            echo "Pull requests must stay source-only."
+            echo "Remove derived files and let main regenerate them after merge."
+            exit 1
+          fi
+
+          if [ "${{ steps.intake.outputs.has_quality_checklist }}" != "true" ]; then
+            echo "PR body must include the Quality Bar Checklist from the template."
+            exit 1
+          fi
+
+          if [ "${{ steps.intake.outputs.has_issue_link }}" != "true" ]; then
+            echo "::notice::No Closes/Fixes issue link detected in the PR body."
+          fi
+
+  source-validation:
+    if: github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    needs: pr-policy
    steps:
      - uses: actions/checkout@v4

@@ -21,9 +74,8 @@ jobs:
        with:
          python-version: "3.10"

-      - name: Install dependencies
-        run: |
-          pip install pyyaml
+      - name: Install Python dependencies
+        run: pip install pyyaml

      - name: Set up Node
        uses: actions/setup-node@v4
@@ -42,21 +94,12 @@ jobs:
          test -f README.md
          test -f CONTRIBUTING.md

-      - name: 🔍 Validate Skills (Soft Mode)
-        run: |
-          npm run validate
+      - name: Validate source changes
+        run: npm run validate

-      - name: 🔗 Validate References
-        run: |
-          npm run validate:references
-
-      - name: 🏗️ Generate Index
-        run: |
-          npm run index
-
-      - name: 📝 Update README
-        run: |
-          npm run readme
+      - name: Validate references
+        if: needs.pr-policy.outputs.requires_references == 'true'
+        run: npm run validate:references

      - name: Audit npm dependencies
        run: npm audit --audit-level=high
@@ -67,67 +110,152 @@ jobs:
          ENABLE_NETWORK_TESTS: "1"
        run: npm run test

-      - name: 📦 Build catalog
+  artifact-preview:
+    if: github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    needs: [pr-policy, source-validation]
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install Python dependencies
+        run: pip install pyyaml
+
+      - name: Set up Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: "lts/*"
+
+      - name: Install npm dependencies
+        run: npm ci
+
+      - name: Generate canonical artifacts preview
+        run: |
+          npm run chain
+          npm run catalog
+
+      - name: Report generated drift
+        run: |
+          managed_files=$(node tools/scripts/generated_files.js --shell --include-mixed)
+          drift_files=$(git diff --name-only -- $managed_files)
+
+          {
+            echo "## Artifact Preview"
+            echo
+            echo "- Primary change: \`${{ needs.pr-policy.outputs.primary_category }}\`"
+            echo "- Categories: \`${{ needs.pr-policy.outputs.categories }}\`"
+            echo "- Derived-file policy: PRs remain source-only; main will canonicalize final generated outputs."
+            echo
+          } >> "$GITHUB_STEP_SUMMARY"
+
+          if [ -z "$drift_files" ]; then
+            echo "No generated drift detected after preview."
+            echo "- Generated drift: none" >> "$GITHUB_STEP_SUMMARY"
+            exit 0
+          fi
+
+          echo "::notice::Generated drift detected in artifact preview."
+          {
+            echo "- Generated drift: detected"
+            echo
+            echo "Predicted file updates:"
+            printf '%s\n' "$drift_files" | sed 's/^/- `/; s/$/`/'
+          } >> "$GITHUB_STEP_SUMMARY"
+
+  main-validation-and-sync:
+    if: github.event_name != 'pull_request'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install Python dependencies
+        run: pip install pyyaml
+
+      - name: Set up Node
+        uses: actions/setup-node@v4
+        with:
+          node-version: "lts/*"
+
+      - name: Install npm dependencies
+        run: npm ci
+
+      - name: Verify directory structure
+        run: |
+          test -d skills/
+          test -d apps/web-app/
+          test -d tools/scripts/
+          test -d tools/lib/
+          test -f README.md
+          test -f CONTRIBUTING.md
+
+      - name: Validate skills
+        run: npm run validate
+
+      - name: Validate references
+        run: npm run validate:references
+
+      - name: Generate index
+        run: npm run index
+
+      - name: Update README
+        run: npm run readme
+
+      - name: Audit npm dependencies
+        run: npm audit --audit-level=high
+        continue-on-error: true
+
+      - name: Run tests
+        env:
+          ENABLE_NETWORK_TESTS: "1"
+        run: npm run test
+
+      - name: Build catalog
        run: npm run catalog

-      - name: Set up GitHub credentials (for auto-sync)
+      - name: Set up GitHub credentials
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
          git config user.name 'github-actions[bot]'
          git config user.email 'github-actions[bot]@users.noreply.github.com'
          git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git

-      - name: Auto-commit registry drift (main only)
+      - name: Auto-commit canonical artifacts
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
-          # If no changes, exit successfully
+          managed_files=$(node tools/scripts/generated_files.js --shell --include-mixed)
+
          git diff --quiet && exit 0

-          # Pull with rebase to integrate remote changes
          git pull origin main --rebase || true
+          git add $managed_files || true

-          git add README.md skills_index.json data/skills_index.json data/catalog.json data/bundles.json data/aliases.json CATALOG.md || true
-
-          # If nothing to commit, exit successfully
          git diff --cached --quiet && exit 0

          git commit -m "chore: sync generated registry files [ci skip]"
          git push origin HEAD

-      - name: ℹ️ Report generated drift (PRs only)
-        if: github.event_name == 'pull_request'
-        run: |
-          if git diff --quiet; then
-            echo "No generated drift detected after validation/build."
-            exit 0
-          fi
-
-          echo "::notice::Generated registry/readme drift detected on this PR."
-          echo "This is informational only on pull requests because main auto-syncs generated artifacts after merge."
-          echo "Files changed by generators:"
-          git diff --name-only
-          {
-            echo "## Generated Drift"
-            echo
-            echo "This PR changes source files that regenerate shared registry artifacts."
-            echo "The drift is allowed on pull requests and will be auto-synced on \`main\` after merge."
-            echo
-            echo "Changed generated files:"
-            git diff --name-only | sed 's/^/- `/; s/$/`/'
-          } >> "$GITHUB_STEP_SUMMARY"
-
-      - name: 🚨 Check for Uncommitted Drift
+      - name: Check for uncommitted drift
        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
        run: |
          if ! git diff --quiet; then
            echo "❌ Detected uncommitted changes produced by registry/readme/catalog scripts."
            echo
            echo "Main must be self-healing after the auto-sync step."
-            echo "To fix locally, run the FULL Validation Chain, then commit and push:"
+            echo "To fix locally, run the canonical maintainer flow:"
+            echo "  npm run release:preflight"
            echo "  npm run chain"
            echo "  npm run catalog"
-            echo "  git add README.md skills_index.json data/skills_index.json data/catalog.json data/bundles.json data/aliases.json CATALOG.md"
-            echo "  git commit -m \"chore: sync generated registry files\""
-            echo "  git push"
+            echo "  git status"
            exit 1
          fi