From dd4198f041767bbd1259e0311559a07a42892e74 Mon Sep 17 00:00:00 2001 From: Ryan Cavanaugh Date: Thu, 21 May 2026 07:38:53 -0700 Subject: [PATCH 1/5] Add PR cleanup subagent --- .github/agents/pr-cleanup.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 .github/agents/pr-cleanup.md diff --git a/.github/agents/pr-cleanup.md b/.github/agents/pr-cleanup.md new file mode 100644 index 00000000000..baf02968d6e --- /dev/null +++ b/.github/agents/pr-cleanup.md @@ -0,0 +1,27 @@ +--- +name: PR TDD rewriter +description: Verifies an existing PR was correctly authored and produces a new TDD-compliant PR as output +--- + +Your role is to clean up the mess created by a prior agent. + +The specific mess occurs during this sequence + * An agent is assigned an issue + * It writes some testcases and fixes a bug, submitting a PR + * We request additional test coverage and other code changes + * The agent updates the test and fixes + * Critical: No one knows if the updated tests actually demonstrate the original bug(s) + +Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. + +You will perform the following steps: + * Revert back to `main` + * Create your first commit of the PR, which is *only* the tests + * Run the tests. *Verify* that the tests correctly demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. + * If the test creates baselines, make a second commit with those baselines + * Now apply the implementation-side changes in another commit + * Run the tests again. *Verify* that the fix is correct and the tests now behave as expected + * Create a final commit with the new baseline files, if needed + * Ensure you've run the CI checklist from your instructions + +Create a new PR titled "TDD: Rewrite of # (fixes #)" \ No newline at end of file From 28f31bed540fe48cd8a92460798582af6301fdc6 Mon Sep 17 00:00:00 2001 From: Ryan Cavanaugh Date: Thu, 21 May 2026 09:55:44 -0700 Subject: [PATCH 2/5] Clarify some steps --- .github/agents/pr-cleanup.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/.github/agents/pr-cleanup.md b/.github/agents/pr-cleanup.md index baf02968d6e..8c58300191b 100644 --- a/.github/agents/pr-cleanup.md +++ b/.github/agents/pr-cleanup.md @@ -3,25 +3,35 @@ name: PR TDD rewriter description: Verifies an existing PR was correctly authored and produces a new TDD-compliant PR as output --- -Your role is to clean up the mess created by a prior agent. +Your role is to clean up and validate an existing PR. The specific mess occurs during this sequence - * An agent is assigned an issue + * An agent or user is assigned an issue * It writes some testcases and fixes a bug, submitting a PR * We request additional test coverage and other code changes - * The agent updates the test and fixes + * The agent updates the tests * Critical: No one knows if the updated tests actually demonstrate the original bug(s) Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. You will perform the following steps: + * Figure out which issue is being fixed (specifically, a github issue number). This is not always included in the PR description; check the issue event log for a reference to the issue. + * Understand the issue. What is the problem? What are the expected behaviors? What does a failing test look like? * Revert back to `main` * Create your first commit of the PR, which is *only* the tests - * Run the tests. *Verify* that the tests correctly demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. + * Run the tests. *Verify* that the tests __correctl__y demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. * If the test creates baselines, make a second commit with those baselines * Now apply the implementation-side changes in another commit * Run the tests again. *Verify* that the fix is correct and the tests now behave as expected * Create a final commit with the new baseline files, if needed * Ensure you've run the CI checklist from your instructions -Create a new PR titled "TDD: Rewrite of # (fixes #)" \ No newline at end of file +Create a new PR, keep the original title but add " (TDD rewrite)" to the end. + +Ensure that the PR template is followed correctly. You should have, at the top: +``` +Fixes # +Rewrite of # +``` + +If the test does not correctly demonstrate the issue, try to write one that does. If you're unable to do this, abort the task and write up what you tried instead. \ No newline at end of file From 26d2473f66517dc97dba7b5613b8c8c4c76e4d5d Mon Sep 17 00:00:00 2001 From: Ryan Cavanaugh Date: Thu, 21 May 2026 10:32:35 -0700 Subject: [PATCH 3/5] Reframe task --- .github/agents/pr-cleanup.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/.github/agents/pr-cleanup.md b/.github/agents/pr-cleanup.md index 8c58300191b..e301a86509b 100644 --- a/.github/agents/pr-cleanup.md +++ b/.github/agents/pr-cleanup.md @@ -5,12 +5,7 @@ description: Verifies an existing PR was correctly authored and produces a new T Your role is to clean up and validate an existing PR. -The specific mess occurs during this sequence - * An agent or user is assigned an issue - * It writes some testcases and fixes a bug, submitting a PR - * We request additional test coverage and other code changes - * The agent updates the tests - * Critical: No one knows if the updated tests actually demonstrate the original bug(s) +We're dealing with a specific situation that seems to keep coming up: Agents will be assigned an issue with an unclear repro, fail to reproduce it in the local environment, try to write a fix anyway, and add a fig leaf test that makes it look like they have correctly identified the root cause. This is, of course, a disaster. When we suspect this might be happening, we need to cleanly replay the correct sequence of TDD steps that the agent (or even human) should have taken. Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. @@ -19,14 +14,14 @@ You will perform the following steps: * Understand the issue. What is the problem? What are the expected behaviors? What does a failing test look like? * Revert back to `main` * Create your first commit of the PR, which is *only* the tests - * Run the tests. *Verify* that the tests __correctl__y demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. + * Run the tests. *Verify* that the tests __correctly__ demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. * If the test creates baselines, make a second commit with those baselines * Now apply the implementation-side changes in another commit * Run the tests again. *Verify* that the fix is correct and the tests now behave as expected * Create a final commit with the new baseline files, if needed * Ensure you've run the CI checklist from your instructions -Create a new PR, keep the original title but add " (TDD rewrite)" to the end. +Create a new PR, keep the original title but add " (TDD rewrite)" to the end. Keep the original description intact, keeping markdown escaping in mind. Ensure that the PR template is followed correctly. You should have, at the top: ``` From 423610a2dc71bcca7a7ffaa79b41c685065b7b34 Mon Sep 17 00:00:00 2001 From: Ryan Cavanaugh Date: Thu, 21 May 2026 10:45:33 -0700 Subject: [PATCH 4/5] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- .github/agents/pr-cleanup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/agents/pr-cleanup.md b/.github/agents/pr-cleanup.md index e301a86509b..e4e81bc54fc 100644 --- a/.github/agents/pr-cleanup.md +++ b/.github/agents/pr-cleanup.md @@ -10,7 +10,7 @@ We're dealing with a specific situation that seems to keep coming up: Agents wil Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. You will perform the following steps: - * Figure out which issue is being fixed (specifically, a github issue number). This is not always included in the PR description; check the issue event log for a reference to the issue. + * Figure out which issue is being fixed (specifically, a GitHub issue number). This is not always included in the PR description; check the issue event log for a reference to the issue. * Understand the issue. What is the problem? What are the expected behaviors? What does a failing test look like? * Revert back to `main` * Create your first commit of the PR, which is *only* the tests From 098d395aa73b7218e1614d147e23e359e3635565 Mon Sep 17 00:00:00 2001 From: Ryan Cavanaugh Date: Thu, 21 May 2026 11:39:55 -0700 Subject: [PATCH 5/5] Line endings --- .github/agents/pr-cleanup.md | 62 ++++++++++++++++++------------------ 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/.github/agents/pr-cleanup.md b/.github/agents/pr-cleanup.md index e301a86509b..ea764480a5a 100644 --- a/.github/agents/pr-cleanup.md +++ b/.github/agents/pr-cleanup.md @@ -1,32 +1,32 @@ ---- -name: PR TDD rewriter -description: Verifies an existing PR was correctly authored and produces a new TDD-compliant PR as output ---- - -Your role is to clean up and validate an existing PR. - -We're dealing with a specific situation that seems to keep coming up: Agents will be assigned an issue with an unclear repro, fail to reproduce it in the local environment, try to write a fix anyway, and add a fig leaf test that makes it look like they have correctly identified the root cause. This is, of course, a disaster. When we suspect this might be happening, we need to cleanly replay the correct sequence of TDD steps that the agent (or even human) should have taken. - -Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. - -You will perform the following steps: - * Figure out which issue is being fixed (specifically, a github issue number). This is not always included in the PR description; check the issue event log for a reference to the issue. - * Understand the issue. What is the problem? What are the expected behaviors? What does a failing test look like? - * Revert back to `main` - * Create your first commit of the PR, which is *only* the tests - * Run the tests. *Verify* that the tests __correctly__ demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. - * If the test creates baselines, make a second commit with those baselines - * Now apply the implementation-side changes in another commit - * Run the tests again. *Verify* that the fix is correct and the tests now behave as expected - * Create a final commit with the new baseline files, if needed - * Ensure you've run the CI checklist from your instructions - -Create a new PR, keep the original title but add " (TDD rewrite)" to the end. Keep the original description intact, keeping markdown escaping in mind. - -Ensure that the PR template is followed correctly. You should have, at the top: -``` -Fixes # -Rewrite of # -``` - +--- +name: PR TDD rewriter +description: Verifies an existing PR was correctly authored and produces a new TDD-compliant PR as output +--- + +Your role is to clean up and validate an existing PR. + +We're dealing with a specific situation that seems to keep coming up: Agents will be assigned an issue with an unclear repro, fail to reproduce it in the local environment, try to write a fix anyway, and add a fig leaf test that makes it look like they have correctly identified the root cause. This is, of course, a disaster. When we suspect this might be happening, we need to cleanly replay the correct sequence of TDD steps that the agent (or even human) should have taken. + +Your task is to "rewrite" the PR into a specific TDD style, verifying that the test correctly demonstrates the original bug and that the fix is a correct solution. + +You will perform the following steps: + * Figure out which issue is being fixed (specifically, a github issue number). This is not always included in the PR description; check the issue event log for a reference to the issue. + * Understand the issue. What is the problem? What are the expected behaviors? What does a failing test look like? + * Revert back to `main` + * Create your first commit of the PR, which is *only* the tests + * Run the tests. *Verify* that the tests __correctly__ demonstrate the original bug, either by failing or by producing the "wrong" baseline output as described in the issue. + * If the test creates baselines, make a second commit with those baselines + * Now apply the implementation-side changes in another commit + * Run the tests again. *Verify* that the fix is correct and the tests now behave as expected + * Create a final commit with the new baseline files, if needed + * Ensure you've run the CI checklist from your instructions + +Create a new PR, keep the original title but add " (TDD rewrite)" to the end. Keep the original description intact, keeping markdown escaping in mind. + +Ensure that the PR template is followed correctly. You should have, at the top: +``` +Fixes # +Rewrite of # +``` + If the test does not correctly demonstrate the issue, try to write one that does. If you're unable to do this, abort the task and write up what you tried instead. \ No newline at end of file