Skip to content

feat: add Copilot CLI governance package#2272

Merged
jackbatzner merged 10 commits into
microsoft:mainfrom
jackbatzner:jackbatzner/agt-plugin
May 16, 2026
Merged

feat: add Copilot CLI governance package#2272
jackbatzner merged 10 commits into
microsoft:mainfrom
jackbatzner:jackbatzner/agt-plugin

Conversation

@jackbatzner
Copy link
Copy Markdown
Collaborator

@jackbatzner jackbatzner commented May 15, 2026

Description

Adds a production-style Copilot CLI governance package (agent-governance-copilot-cli) plus a runnable example (examples/copilot-cli-agt) for AGT-backed policy enforcement in Copilot CLI.

This PR:

  • adds explicit agt-copilot install / update / uninstall / doctor / policy commands instead of npm postinstall side effects
  • ships a packaged Copilot extension with default, balanced, strict, and advisory policy profiles
  • hardens the runtime against unsafe SDK overrides, weak custom policy downgrades, tool-output poisoning blind spots, and direct shell-based secret / metadata access
  • updates docs and example flows so the package is the supported install path and the example remains the tutorial/reference path
  • fixes CI / publish / ESRP workflow gaps for package coverage, permissions scoping, concurrency behavior, and Node version handling

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Maintenance (dependency updates, CI/CD, refactoring)
  • Security fix

Package(s) Affected

  • agent-os-kernel
  • agent-mesh
  • agent-runtime
  • agent-sre
  • agent-governance
  • docs / root

Checklist

  • My code follows the project style guidelines (ruff check)
  • I have added tests that prove my fix/feature works
  • All new and existing tests pass (pytest)
  • I have updated documentation as needed
  • I have signed the Microsoft CLA

Attribution & Prior Art

  • This contribution does not contain code copied or derived from other projects without attribution
  • Any external projects that inspired this design are credited in code comments or documentation
  • If this PR implements functionality similar to an existing open-source project, I have listed it below

Prior art / related projects (if any):

AI Assistance

  • I can explain every meaningful change in this PR: what it does, why, and what tradeoffs were considered
  • I have run tests and verification appropriate for this change
  • No part of this PR was autonomously submitted by an AI agent without my review
  • I have not used AI to generate review comments on others' PRs

If AI tools materially shaped this change, briefly note what was used:

  • GitHub Copilot CLI / Copilot were used for implementation drafting, review iteration, and test scaffolding; all changes were reviewed and edited before push.

IP, Patents, and Licensing

  • This contribution does not implement patent-pending or patent-encumbered techniques
  • This contribution does not require an NDA or licensing agreement to understand or use
  • Any AI tools used have terms compatible with the MIT License

Related Issues

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

PR Review Summary

Check Status Details
🔍 Code Review ❌ Failed Issues detected
🛡️ Security Scan ✅ Passed No issues found
🔄 Breaking Changes ✅ Passed No issues found
📝 Docs Sync ✅ Completed Analysis complete
🧪 Test Coverage ✅ Completed Analysis complete

Verdict: ❌ Changes needed

@github-actions github-actions Bot added documentation Improvements or additions to documentation scripts/ci/cd size/XL Extra large PR (500+ lines) labels May 15, 2026
@jackbatzner jackbatzner requested a review from Copilot May 15, 2026 19:40
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

TL;DR: 2 blockers, 1 warning. Fix #1 and #2 and this ships.

# Sev Issue Where
1 Block bin/agt-copilot.mjs is referenced but missing (breaks npx + npm run check) agent-governance-copilot-cli/package.json
2 Block realpathSync runs before existence checks, causing hard failure instead of a controlled “SDK not found” error agent-governance-copilot-cli/.../lib/sdk-loader.mjs
3 Warn npm-agentmesh-mcp-proxy selector isn’t whitelisted in Python resolver’s NPM_INPUTS, producing an “Unknown package” error annotation .github/workflows/publish.yml

#1: Add the missing bin/agt-copilot.mjs entrypoint (or update bin/scripts to point at the real entrypoint).
#2: Make SDK path canonicalization tolerant of missing files (defer/wrap realpathSync so missing vendored SDK yields a readable error).

Changes:

  • Adds a production-style Copilot CLI governance installer package (@microsoft/agent-governance-copilot-cli) plus a runnable example and scenario corpus.
  • Adds hardened policy profiles/default policy for Copilot CLI prompt/tool/tool-output governance.
  • Updates docs + CI/publish/ESRP plumbing to include the new package and Node 22 handling.

Reviewed changes

Copilot reviewed 48 out of 50 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
README.md Adds Copilot CLI surface + install row
mkdocs.yml Adds Copilot CLI package + tutorial nav entries
examples/copilot-cli-agt/README.md Documents example + production install path
examples/copilot-cli-agt/package.json Example Node test/check scripts
examples/copilot-cli-agt/test/policy-engine.test.mjs Example policy/runtime alignment tests
examples/copilot-cli-agt/scripts/install-extension.sh Bash install/update wrapper
examples/copilot-cli-agt/scripts/install-extension.ps1 PowerShell install/update wrapper
examples/copilot-cli-agt/config/default-policy.json Example default policy baseline
examples/copilot-cli-agt/config/profiles/strict.json Example strict profile
examples/copilot-cli-agt/config/profiles/balanced.json Example balanced profile
examples/copilot-cli-agt/config/profiles/advisory.json Example advisory profile
examples/copilot-cli-agt/.github/extensions/agt-global-policy/package.json Example extension module metadata
examples/copilot-cli-agt/.github/extensions/agt-global-policy/extension.mjs Example extension entrypoint
examples/copilot-cli-agt/.github/extensions/agt-global-policy/main.mjs Example Copilot CLI hooks/tools wiring
examples/copilot-cli-agt/.github/extensions/agt-global-policy/lib/sdk-loader.mjs Example SDK loader
examples/copilot-cli-agt/.github/extensions/agt-global-policy/lib/poisoning.mjs Example text flatten/summarize helpers
examples/copilot-cli-agt/scenarios/guarded-repo-triage/README.md Scenario walkthrough
examples/copilot-cli-agt/scenarios/guarded-repo-triage/expected-outcomes.md Expected outcomes
examples/copilot-cli-agt/scenarios/guarded-repo-triage/proof-corpus.json Machine-readable proof corpus
examples/copilot-cli-agt/scenarios/guarded-repo-triage/proof-package.md Evidence checklist
examples/copilot-cli-agt/scenarios/guarded-repo-triage/tool-output/poisoned-web-content.txt Poisoned output sample
examples/copilot-cli-agt/scenarios/guarded-repo-triage/prompts/prompt-injection.txt Prompt injection sample
examples/copilot-cli-agt/scenarios/guarded-repo-triage/prompts/unsafe-bootstrap.txt Unsafe bootstrap sample
docs/tutorials/46-copilot-cli-governance.md New Copilot CLI installer tutorial
docs/packages/index.md Fixes TS SDK install + adds Copilot CLI row
docs/packages/copilot-cli-governance.md New package docs page
docs/index.md Fixes TS SDK install command
agent-governance-typescript/package-lock.json Bumps TS SDK lock version to 3.6.0
agent-governance-copilot-cli/README.md Package README
agent-governance-copilot-cli/package.json New npm package manifest (bin/scripts)
agent-governance-copilot-cli/package-lock.json New npm lockfile
agent-governance-copilot-cli/test/policy-engine.test.mjs Packaged baseline policy/runtime tests
agent-governance-copilot-cli/test/install.test.mjs Installer/doctor/policy command tests
agent-governance-copilot-cli/assets/extensions/agt-global-policy/package.json Packaged extension module metadata
agent-governance-copilot-cli/assets/extensions/agt-global-policy/extension.mjs Packaged extension entrypoint
agent-governance-copilot-cli/assets/extensions/agt-global-policy/main.mjs Packaged Copilot CLI hooks/tools wiring
agent-governance-copilot-cli/assets/extensions/agt-global-policy/lib/sdk-loader.mjs Packaged hardened SDK loader
agent-governance-copilot-cli/assets/extensions/agt-global-policy/lib/poisoning.mjs Packaged poisoning helpers (+ windows)
agent-governance-copilot-cli/assets/extensions/agt-global-policy/config/default-policy.json Packaged default policy
agent-governance-copilot-cli/assets/extensions/agt-global-policy/config/profiles/strict.json Packaged strict profile
agent-governance-copilot-cli/assets/extensions/agt-global-policy/config/profiles/balanced.json Packaged balanced profile
agent-governance-copilot-cli/assets/extensions/agt-global-policy/config/profiles/advisory.json Packaged advisory profile
.github/workflows/ci.yml CI path filters + Node version matrix updates
.github/workflows/publish.yml Adds npm matrix + permissions scoping
.github/pipelines/esrp-publish.yml Adds package entries + per-package Node version
.github/dependabot.yml Adds dependabot for new npm package
.github/CODEOWNERS Adds codeowners for new package path
Files not reviewed (2)
  • agent-governance-copilot-cli/package-lock.json: Language not supported
  • agent-governance-typescript/package-lock.json: Language not supported

Comment thread agent-governance-copilot-cli/package.json
Comment thread .github/workflows/publish.yml
imran-siddique
imran-siddique previously approved these changes May 15, 2026
Copy link
Copy Markdown
Member

@imran-siddique imran-siddique left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Legit feature add — Copilot CLI governance package with policy profiles. CI failures are pre-existing test config drift and process gates, not caused by this PR. Merging per trusted contributor workflow.

@imran-siddique imran-siddique marked this pull request as ready for review May 15, 2026 20:36
@imran-siddique imran-siddique enabled auto-merge (squash) May 15, 2026 20:36
jackbatzner and others added 7 commits May 15, 2026 18:40
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

🤖 AI Agent: breaking-change-detector — API Compatibility

API Compatibility

No breaking changes detected.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

🤖 AI Agent: docs-sync-checker — Docs Sync

Docs Sync

  • agent-governance-copilot-cli -- missing docstring for new public APIs
  • README.md -- section on installation and usage needs update to reflect new governance package
  • CHANGELOG.md -- missing entry for added Copilot CLI governance package and behavioral changes

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

🤖 AI Agent: security-scanner — View details

No security issues found.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

🤖 AI Agent: test-generator — `agent-governance-copilot-cli/index.ts`

agent-governance-copilot-cli/index.ts

  • test_policy_command_execution -- Validate that all policy subcommands (install, update, uninstall, doctor) execute correctly with expected outputs.
  • test_policy_profile_loading -- Ensure default, balanced, strict, and advisory policy profiles load and apply correctly.
  • test_unsafe_sdk_override_protection -- Verify runtime blocks unsafe SDK overrides.
  • test_tool_output_poisoning -- Test detection and handling of poisoned tool outputs.
  • test_direct_shell_access_restriction -- Confirm direct shell-based secret/metadata access is restricted.

examples/copilot-cli-agt/example.ts

  • test_example_policy_enforcement -- Validate example demonstrates correct AGT-backed policy enforcement.
  • test_example_error_handling -- Ensure example handles policy violations and errors gracefully.

agent-governance-copilot-cli/utils.ts

  • test_parse_policy_config -- Validate parsing of policy configuration files, including malformed cases.
  • test_node_version_handling -- Ensure correct handling of Node version mismatches during runtime.

agent-governance-copilot-cli/security.ts

  • test_custom_policy_downgrade_protection -- Verify runtime prevents weak custom policy downgrades.
  • test_runtime_hardening -- Test all runtime hardening measures (e.g., sandboxing, dependency validation).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 15, 2026

🤖 AI Agent: code-reviewer — Action Items:

TL;DR: 2 blockers, 1 warning. The PR introduces a critical security gap and a CI/CD misconfiguration; minor documentation issue can be addressed later.

# Sev Issue Where
1 🔴 Potential bypass of policy enforcement due to unsafe SDK override handling. agent-governance-copilot-cli package
2 🔴 CI/CD pipeline misconfiguration: nodeVersion fallback logic may fail. .github/pipelines/esrp-publish.yml
3 🟡 Missing test coverage for new agt-copilot commands. agent-governance-copilot-cli package

Action Items:

  1. Fix unsafe SDK override handling to ensure policy enforcement cannot be bypassed.
  2. Correct nodeVersion fallback logic in .github/pipelines/esrp-publish.yml.

Warnings:

# Issue Where Follow-up?
1 Add test coverage for agt-copilot commands to ensure reliability. agent-governance-copilot-cli package

Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
Signed-off-by: Jack Batzner <jackbatzner@microsoft.com>
@jackbatzner
Copy link
Copy Markdown
Collaborator Author

I went through the open bot threads and handled the actionable ones.

Fixed in follow-up commits:

  • deferred vendored SDK realpathSync() until after the existence check so missing/corrupt vendored SDKs now fall through to the intended “Unable to locate...” error instead of throwing ENOENT
  • added npm-agentmesh-mcp-proxy to NPM_INPUTS in publish.yml so npm-only selector values no longer get flagged as unknown by the Python matrix resolver

The earlier package.json comment about bin/agt-copilot.mjs is now stale — that entrypoint is present, tracked, and wired correctly.

At this point the only remaining red item I’m seeing is the maintainer-review policy gate; the code-side PR feedback has been addressed.

@jackbatzner
Copy link
Copy Markdown
Collaborator Author

@imran-siddique , pushed updates for failing checks in the PR. Also addressed AI review feedback. Would appreciate a second review! Thanks in advance

@jackbatzner jackbatzner disabled auto-merge May 16, 2026 01:12
@jackbatzner jackbatzner enabled auto-merge (squash) May 16, 2026 01:12
@jackbatzner jackbatzner merged commit 5477488 into microsoft:main May 16, 2026
127 of 128 checks passed
MohammadHaroonAbuomar pushed a commit to MohammadHaroonAbuomar/agt-acs that referenced this pull request Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation scripts/ci/cd size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants