fix(ci): verify regenerated patch & report resolution honestly (#187)

## Why

While reviewing the auto-generated patch PRs (#184, #185, #186) they
*looked* like failed AI resolutions — identical patch content, body
claiming "conflicts were resolved with AI". Investigation showed the
opposite: the existing patches applied **cleanly**, the AI step was
**skipped**, and the byte-identical output is correct (every source file
the patches touch is unchanged between the old and new Node releases).

The real defects were **visibility and missing guards**, not the
resolver:

- The PR body *always* said conflicts "were resolved with
`actions/ai-inference@v2`", even when AI never ran — making legitimate
no-op version bumps look bogus.
- A quota-exhausted / rate-limited AI response (empty body) surfaced as
a vague `SEARCH block not found` instead of a clear diagnosis.
- A fully-upstreamed patch could regenerate to an empty diff and ship an
unpatched build.

## Changes

`.github/workflows/patch-node.yml`
- **Honest PR body** — reports whether AI actually ran vs. clean apply,
whether the patch changed vs. is an identical carry-forward, and the
sanity-check result. Uses a random heredoc delimiter so a body line
can't terminate the env block early.
- **Empty-diff abort** — if the regenerated patch is zero bytes (e.g.
every hunk landed upstream), abort instead of shipping an unpatched
build.
- **"Sanity-check generated patch" step** — re-applies the generated
diff to a pristine tree. This is a *regression guard on the
diff-generation plumbing* (`--src-prefix`/`-p1`), **not** proof of
semantic correctness: any tree round-trips through `git diff`/`git
apply`, so only the build (`test-patch.yml`) validates that a resolution
is actually right.

`.github/scripts/ai_resolver.py`
- Empty AI response while conflicts are outstanding now fails with an
explicit **"GitHub Models quota exhausted or rate-limited"** message and
exits 1, so PR creation is skipped.

## Verification
- `python3 -m py_compile` + YAML parse pass.
- Empty-response guard tested locally: exits 1 with the quota
diagnostic.
- Existing safe-abort behaviour (incomplete AI resolution → PR skipped)
is unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Author: robertsLando

.github/scripts/ai_resolver.py

@@ -288,6 +288,20 @@ def cmd_apply(node_dir: str, response_path: str) -> None:
     initial_rejects = find_reject_files(node_dir)
     total = len(initial_rejects)
 
+    # An empty model response while conflicts are still outstanding almost
+    # always means the GitHub Models quota is exhausted or the request was
+    # rate-limited — the action can return a 200 with no content. Fail loudly
+    # with that diagnosis instead of letting it surface downstream as a vague
+    # "SEARCH block not found", and never let an empty resolution reach a PR.
+    if total and not response.strip():
+        print("❌ AI response is empty — the model returned no content.")
+        print("   Likely cause: GitHub Models quota exhausted or request rate-limited.")
+        print("CONFLICTS_RESOLVED=0")
+        print(f"TOTAL_CONFLICTS={total}")
+        print("HAS_UNRESOLVED=True")
+        print(f"FAILED_FILES={' '.join(sorted(os.path.relpath(r[:-4], node_dir) for r in initial_rejects))}")
+        sys.exit(1)
+
     # Count the hunks the AI must resolve per file BEFORE applying anything.
     # Hunks upstream already landed are excluded — the file already carries
     # their post-image, so no SEARCH/REPLACE block is expected (or possible)

.github/workflows/patch-node.yml

@@ -118,36 +118,101 @@ jobs:
         run: |
           set -eo pipefail
           # Python exits non-zero on unresolved conflicts; pipefail+set -e
-          # ensures the whole step fails so the PR step is skipped.
+          # ensures the whole step fails so the PR step is skipped. The summary
+          # is saved to a temp file that the verify step folds into the PR body.
           python3 .github/scripts/ai_resolver.py apply ../node "${{ steps.ai.outputs.response-file }}" \
             | tee "$RUNNER_TEMP/resolution_output.txt"
-          {
-            echo "RESOLUTION_OUTPUT<<EOF"
-            cat "$RUNNER_TEMP/resolution_output.txt"
-            echo "EOF"
-          } >> "$GITHUB_ENV"
 
       - name: Generate new patch & update patches.json
         run: |
           set -e
 
+          NEW_PATCH="patches/node.v${{ inputs.nodeVersion }}.cpp.patch"
+
           cd ../node
           git add -A
           git diff --staged --src-prefix=node/ --dst-prefix=node/ \
-            > "../pkg-fetch/patches/node.v${{ inputs.nodeVersion }}.cpp.patch"
+            > "../pkg-fetch/$NEW_PATCH"
 
           cd ../pkg-fetch
+
+          # A zero-byte diff means the resolved tree carries no change at all —
+          # the patch would be a no-op and the build would be unpatched. This
+          # often means every hunk has landed upstream and the patch is no
+          # longer needed; either way a human should look, so never open a PR.
+          if [ ! -s "$NEW_PATCH" ]; then
+            echo "❌ Regenerated patch is empty — the resolved tree produced no diff."
+            echo "   Every hunk may have landed upstream in v${{ inputs.nodeVersion }}; investigate manually. Aborting."
+            exit 1
+          fi
+
+          # Record whether the regenerated patch is byte-identical to the source
+          # patch. That is legitimate (the touched source files were unchanged
+          # between releases) but worth surfacing in the PR body so a no-op
+          # carry-forward is not mistaken for a failed resolution.
+          if [ -f "$PATCH_FILE" ] && cmp -s "$PATCH_FILE" "$NEW_PATCH"; then
+            echo "PATCH_CHANGED=false" >> "$GITHUB_ENV"
+          else
+            echo "PATCH_CHANGED=true" >> "$GITHUB_ENV"
+          fi
+          echo "NEW_PATCH=$NEW_PATCH" >> "$GITHUB_ENV"
+
           # Replace the old version entry with the new one. If a `patchFile`
           # input was supplied, $PATCH_VERSION refers to that source patch.
           sed -i \
             "s/\"v$PATCH_VERSION\": \[\"node.v$PATCH_VERSION.cpp.patch\"\]/\"v${{ inputs.nodeVersion }}\": \[\"node.v${{ inputs.nodeVersion }}.cpp.patch\"\]/" \
             patches/patches.json
 
           # Remove the old patch file unless we just overwrote it in place.
-          if [ "$PATCH_FILE" != "patches/node.v${{ inputs.nodeVersion }}.cpp.patch" ]; then
+          if [ "$PATCH_FILE" != "$NEW_PATCH" ]; then
             rm -f "$PATCH_FILE"
           fi
 
+      - name: Sanity-check generated patch & summarize
+        run: |
+          set -e
+
+          # Regression guard on the diff-generation flags above: re-apply the
+          # generated patch to a pristine v${{ inputs.nodeVersion }} tree. This is a cheap check that
+          # the `--src-prefix`/`-p1` plumbing still lines up — it does NOT prove
+          # the resolution is semantically correct (any tree round-trips through
+          # `git diff`/`git apply`); only the build (test-patch.yml) does that.
+          cd ../node
+          git reset --hard HEAD >/dev/null
+          git clean -fd >/dev/null   # drop leftover .rej/.orig
+          if git apply --check "../pkg-fetch/$NEW_PATCH"; then
+            VERIFY="✅ Generated patch re-applies to a pristine v${{ inputs.nodeVersion }} tree (build still validates semantics)."
+          else
+            echo "❌ Generated patch does NOT re-apply to a pristine v${{ inputs.nodeVersion }} tree — diff-generation plumbing is broken. Aborting before PR."
+            exit 1
+          fi
+          cd ../pkg-fetch
+
+          # Compose an honest PR body. Never claim the AI ran when it did not.
+          if [ "${{ steps.apply.outputs.needs_ai }}" = "true" ]; then
+            MODE="🤖 Conflicts were resolved with \`actions/ai-inference@v2\` (GitHub Models)."
+            if [ -f "$RUNNER_TEMP/resolution_output.txt" ]; then
+              MODE="$MODE"$'\n\n'"$(cat "$RUNNER_TEMP/resolution_output.txt")"
+            fi
+          else
+            MODE="✅ The existing v$PATCH_VERSION patch applied cleanly to v${{ inputs.nodeVersion }} — no AI resolution was needed."
+          fi
+
+          if [ "$PATCH_CHANGED" = "false" ]; then
+            CHANGED="ℹ️ The regenerated patch is byte-identical to v$PATCH_VERSION — the source files it touches did not change between the two releases."
+          else
+            CHANGED="ℹ️ The patch was updated to match the v${{ inputs.nodeVersion }} source tree."
+          fi
+
+          # Random delimiter so a body line that happens to equal the marker
+          # can't terminate the heredoc early.
+          DELIM="RESOLUTION_EOF_$(uuidgen)"
+          {
+            echo "RESOLUTION_OUTPUT<<$DELIM"
+            printf '%s\n\n%s\n\n%s\n' "$MODE" "$CHANGED" "$VERIFY"
+            echo "$DELIM"
+          } >> "$GITHUB_ENV"
+
       - name: Create Pull Request
         uses: peter-evans/create-pull-request@v8
         with:
@@ -157,12 +222,9 @@ jobs:
           body: |
             ## Node.js Patch Update to v${{ inputs.nodeVersion }}
 
-            This PR updates the Node.js patch to version ${{ inputs.nodeVersion }}.
-
-            Patch conflicts (if any) were resolved with `actions/ai-inference@v2`
-            using GitHub Models — no third-party API key required.
+            This PR updates the Node.js patch from v${{ env.PATCH_VERSION }} to v${{ inputs.nodeVersion }}.
 
-            ${{ env.RESOLUTION_OUTPUT && '### AI Resolution Details' || '' }}
+            ### Resolution
             ${{ env.RESOLUTION_OUTPUT }}
           branch: "nodejs-v${{ inputs.nodeVersion }}"
           base: "main"