Add canvas extensibility support#1372
Open
jmoseley wants to merge 15 commits into
Open
Conversation
This is a hand-applied snapshot of the Rust SDK changes for canvas
extensibility V1, lifted from a vendored copy in the github-app
repository (which had been edited in-place while this upstream branch
was not yet ready).
IMPORTANT CAVEAT: In this repo, most of the Rust types here are
normally generated from TypeScript schemas via codegen. The
corresponding TypeScript schema changes were authored on another
agent's branch (0ebfff40f3) that has not been pushed to origin.
Specifically, that branch contains:
- manifest \`agentActions\`
- \`session.canvas.*\` host SDK
- canvas-level \`inputSchema\`
- \`canvas_input_invalid\` validation
- \`actionId\` -> \`actionName\` rename
- removal of \`requires\` / \`extensionRequires\`
Because of this, this branch is a Rust-only snapshot of the desired
end state, not the full TypeScript-driven implementation. Whoever
picks this up will need to either:
(a) merge the TypeScript work from the orchestrator's branch and
re-run codegen, or
(b) treat this Rust diff as a spec and re-derive the TypeScript
schema from it.
Build status on this snapshot (cargo, in rust/):
- \`cargo check\`: passes
- \`cargo test --no-run\`: fails to compile some test binaries
(protocol_version_test, session_test). These are pre-existing
integration tests that rely on APIs not yet adjusted to match
this snapshot (e.g. \`Client::from_streams_with_trace_provider\`).
Intentionally not fixed here -- the goal of this branch is to
preserve the snapshot, not to make it green standalone.
Branched off 477834f ("Publish .snupkg symbols package to NuGet.org
(#1345)") -- the SHA the vendor was synced from -- rather than current
main, so the diff applies cleanly. Rebase onto main as needed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Additive Rust SDK changes for V1.1 canvas extensibility (mirrors the
locked TypeScript wire shape on the runtime side).
New `canvas` module (rust/src/canvas.rs):
- `CanvasDeclaration`, `CanvasAgentActionDeclaration`,
`CanvasToolbarItemDeclaration` — wire shape mirroring runtime
commit 0d9535192b on jmoseley/canvas-runtime-support
(copilot-agent-runtime).
- `CanvasHandler` trait + `Canvas` / `CanvasBuilder` ergonomics.
- `CanvasOpenContext`, `CanvasActionContext`, `CanvasInitContext`,
`CanvasOpenResponse`, `CanvasError` request/response types.
- `CanvasRegistry` + `build_registry` + `CanvasInvokeParams` +
`dispatch_canvas_invoke` routing implementation.
- 11 unit tests covering serialization, registry routing, dispatch
semantics.
`SessionConfig` / `ResumeSessionConfig` (rust/src/types.rs):
- New `canvases: Vec<Canvas>` field — provider declaration. Empty
default; skips serialize when empty; never deserializes (handlers
are non-Serde types).
- New `request_canvas_renderer: Option<bool>` field — renderer-side
opt-in mirroring `request_elicitation`. Default None; when true,
runtime surfaces canvas agent tools (`open_canvas`,
`discover_canvases`, ...) to the model.
- Matching `with_canvases` / `with_request_canvas_renderer` builders,
Debug fields, and Default ctor entries.
- `HostCapabilitiesConfig.canvas` doc-comment updated to call out
renderer-only semantics (kept during transition; will be deleted
once V1.1 finalizes).
Dispatch wiring (rust/src/session.rs):
- `create_session` / `resume_session` build `Arc<CanvasRegistry>`
from `config.canvases` and thread it through `spawn_event_loop`
to `handle_request`.
- `hostExtension.invoke` arm intercepts inner
`method == "canvas.action.invoke"`, deserializes
`CanvasInvokeParams`, dispatches via `dispatch_canvas_invoke`,
wraps result in `HostedExtensionResponse::{Success, Error}`.
Falls through to legacy `on_hosted_extension` for non-canvas
hostExtension calls.
`pub mod canvas` re-exported from `lib.rs`.
Validation: cargo test --lib in rust/ passes 147/147 (no
regressions; new canvas tests included in count). Cargo check
clean.
This commit is additive — no breaking changes. Legacy
`HostedExtension*` / `host_extensions` / `request_host_extension` /
`HostCapabilitiesConfig` paths remain functional during the V1→V1.1
transition. They will be deleted once runtime ships slices E + F and
github-app integration testing confirms the new path works
end-to-end.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…derer upstream) Runtime PR #8441 commit 11e040dc1b renamed the renderer-capability gate from hostCapabilities.canvas to a top-level requestCanvasRenderer field on SessionCreate/Resume, mirroring requestElicitation. The Rust SDK already has request_canvas_renderer wired, so the legacy field + struct are now dead. - Delete HostCapabilitiesConfig struct. - Drop host_capabilities field from SessionConfig + ResumeSessionConfig (including Default impls). - Update serialization tests to drop hostCapabilities assertions. 147/147 lib tests pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-up Slice F of the V1.1 canvas extensibility cutover. Adds the Node-side counterpart to the Rust SDK Canvas/CanvasBuilder shape (commit f095993) so extensions can declare canvases dynamically via joinSession instead of a static copilot-extension.json manifest. Surface (exported from both '@github/copilot-sdk' and '@github/copilot-sdk/extension'): - createCanvas(options): Canvas — packages a CanvasDeclaration with in-process onOpen / onAction / onFocus / onClose / onReload closures. - CanvasDeclaration, CanvasAgentActionDeclaration, CanvasToolbarItemDeclaration — wire shape (mirrors copilot-agent-runtime src/core/protocol/types.ts on jmoseley/canvas-runtime-support@0d9535192b). - CanvasOpenContext, CanvasActionContext, CanvasLifecycleContext, CanvasOpenResponse, CanvasOptions, CanvasError. Wire-up: - SessionConfig + ResumeSessionConfig gain canvases?: Canvas[] and requestCanvasRenderer?: boolean. - client.ts createSession / resumeSession serialize canvases.map(c => c.declaration) onto the session.create / session.resume RPC and pass through requestCanvasRenderer. - CopilotSession gains a per-session canvas registry (registerCanvases / getCanvas) parallel to the existing toolHandlers registry. - client.ts registers a 'hostExtension.invoke' onRequest handler that intercepts inner method === 'canvas.action.invoke', routes by (canvasId, actionName) to the registered Canvas's handlers, and surfaces CanvasError as the wire error envelope. Other inner methods are rejected — no other hostExtension.invoke variants are in use post-V1.1. Example: import { joinSession, createCanvas } from '@github/copilot-sdk/extension'; const counter = createCanvas({ id: 'counter', onOpen: async (ctx) => ({ url: 'http://localhost:3000' }), onAction: async (ctx) => ({ value: 1 }), }); await joinSession({ canvases: [counter] }); typecheck clean; 146/147 unit tests pass (1 pre-existing skip). Build emits dist/canvas.{js,d.ts} + dist/cjs equivalents and re-exports createCanvas from dist/extension.{js,d.ts} and dist/index.{js,d.ts}. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
V1.1 canvas dispatch is now the only path through `hostExtension.invoke`.
This deletes all transitional types and the `on_hosted_extension` trait
method.
Removed types:
- ExtensionRegistrationConfig + host_extensions / extension_roots
- HostedExtensionInvokeRequest / HostedExtensionRequest
- HostedExtensionResponse + Success/Error variants + helper
- HostedExtensionError
Removed config fields:
- SessionConfig.extension_registrations / .request_host_extension
- ResumeSessionConfig.extension_registrations / .request_host_extension
Removed handler surface:
- HandlerEvent::HostedExtension
- HandlerResponse::HostedExtension
- SessionHandler::on_hosted_extension default trait method
- NoopHandler HostedExtension match arm
The `hostExtension.invoke` JSON-RPC handler in session.rs now only
accepts `canvas.action.invoke` inner method; everything else returns a
structured `unsupported_method` error. Response JSON is constructed
inline (`{ ok: true, result: \... }` / `{ ok: false, error: { code, message } }`).
143/143 lib tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`std::mem::take(&mut config.canvases)` ran before `serde_json::to_value(&config)`, leaving the field empty so `skip_serializing_if = Vec::is_empty` dropped it from the JSON-RPC payload entirely. The comment claimed `Canvas::serialize` would still emit the declarations, but the vec had already been moved out. `Canvas::serialize` delegates to `CanvasDeclaration` (handlers are not part of the wire shape), so we can just build the registry from `&config.canvases` and let serde walk the live vec. Applies to both `create_session` and `resume_session`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirrors the runtime's new SessionCreateRequest.requestExtensions field (github/copilot-agent-runtime#8441, commit 3029ce07cf). When set on session.create / session.resume, the runtime wires extension management tools (extensions_reload, extensions_manage) and per-extension tool dispatch onto the session for this connection. Requires the runtime to have the EXTENSIONS experimental feature flag enabled; otherwise the runtime silently skips wiring even when the flag is true (kill-switch semantics preserved). Rust: SessionConfig + ResumeSessionConfig new request_extensions field with builder + Debug + Default. Node: SessionConfig field, ResumeSessionConfig Pick passthrough, client.ts wire passthrough for both create and resume. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirrors runtime PR #8441 making agent-supplied instance_id required on canvas.open. Handlers now receive ctx.instance_id directly instead of generating their own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
handleHostExtensionInvoke was returning JSON-RPC-style {id, result} /
{id, error}, but runtime expects HostedExtensionResponse envelope
{ok: true, result} / {ok: false, error} per protocol/types.ts. This
caused all extension-provided canvases to fail with
canvas_invoke_malformed_response after the runtime added its
defensive guard.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Threads agent-canvas instance rehydrate from the host through both Rust and Node SDKs to the runtime's session.resume RPC. Rust: - New CanvasInstanceRehydrate struct in canvas.rs (camelCase serde). - ResumeSessionConfig gains open_canvas_instances: Vec<CanvasInstanceRehydrate> with a with_open_canvas_instances() builder; serializes via the existing serde_json::to_value(&config) wire path in resume_session. - Debug impl includes the new field. Node: - CanvasInstanceRehydrate interface mirrored in canvas.ts, re-exported from index.ts. - ResumeSessionConfig.openCanvasInstances?: CanvasInstanceRehydrate[]. - client.ts session.resume payload forwards the field. The runtime side (copilot-agent-runtime PR #8441) consumes this via SessionResumeRequest.openCanvasInstances and rehydrateCanvasInstances(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jmoseley
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds SDK-side Canvas extensibility (V1 + V1.1) across Rust and Node.js, wiring session config, RPC surfaces, and host-side dispatch so hosts can declare canvases and handle canvas.action.invoke callbacks.
Changes:
- Introduces Canvas declaration + handler APIs (Rust
canvasmodule; NodecreateCanvas) and threads canvas declarations throughsession.create/session.resume. - Adds session opt-in flags for canvas renderer + extension surface (
request_canvas_renderer/request_extensions) and resume support for rehydrating open canvas instances. - Extends tool/permission invocation metadata with optional
namespace/canvasId(andextensionIdwhere applicable) and adds generated RPC/types for new canvas/extension methods.
Show a summary per file
| File | Description |
|---|---|
| rust/tests/api_types_test.rs | Updates extension test helper for new Extension field(s). |
| rust/src/types.rs | Adds session config fields (canvas/extension opts, canvases, open_canvas_instances) and extends tool/permission invocation metadata. |
| rust/src/tool.rs | Updates examples/tests to account for new tool invocation fields. |
| rust/src/session.rs | Builds/retains a per-session canvas registry and adds hostExtension.invoke canvas dispatch path. |
| rust/src/lib.rs | Exposes new canvas module and extends Error::Rpc with optional structured data. |
| rust/src/handler.rs | Updates handler tests for expanded ToolInvocation shape. |
| rust/src/generated/session_events.rs | Regenerates session-event payloads to include namespace/canvas metadata and extension tool routing fields. |
| rust/src/generated/rpc.rs | Adds session.canvas.* RPC namespace and session.extensions.discoverCanvases. |
| rust/src/generated/api_types.rs | Regenerates protocol types/constants for canvas lifecycle/invoke and extension/canvas discovery details. |
| rust/src/canvas.rs | New Rust Canvas V1.1 API: declarations, handler trait, registry, and dispatch implementation + tests. |
| nodejs/src/types.ts | Adds canvas/extension session config fields and resume openCanvasInstances typing. |
| nodejs/src/session.ts | Adds per-session canvas registry (register/get) for dispatch. |
| nodejs/src/index.ts | Re-exports Canvas APIs from the main Node entrypoint. |
| nodejs/src/extension.ts | Re-exports Canvas APIs for extension consumers. |
| nodejs/src/client.ts | Threads canvases + opt-in flags through create/resume payloads; adds hostExtension.invoke dispatcher. |
| nodejs/src/canvas.ts | New Node Canvas V1.1 API: declarations, handler closures, and dispatch helper. |
Copilot's findings
- Files reviewed: 13/16 changed files
- Comments generated: 5
Rewrites doc comments on the canvas declarations, session config fields, and dispatch wiring to describe the surface as-is without versioning narrative or references to the removed hosted-extension types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…me-support # Conflicts: # nodejs/src/types.ts
This comment has been minimized.
This comment has been minimized.
- Rust+Node canvas dispatch: require instanceId for lifecycle verbs and
custom actions (was silently defaulting to empty string).
- Rust hostExtension.invoke: validate envelope.session_id matches the
session handling the request; return a structured session_mismatch
error envelope on mismatch.
- Node handleHostExtensionInvoke: return { ok:false, error } envelopes
for invalid payloads, missing sessions, and unsupported inner methods
instead of throwing (which would have surfaced as JSON-RPC transport
errors and broken the runtime-side contract).
- Re-export CanvasInstanceRehydrate from @github/copilot-sdk/extension
so extension consumers can type ResumeSessionConfig.openCanvasInstances
without reaching into internal paths.
- Fix canvas test that called canvas.open without instance_id.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
Contributor
Author
|
Thanks — confirming intentional scope. Canvas extensibility is shipping Node + Rust first because those are the two SDKs the immediate consumers (Copilot CLI runtime, Tauri app, and the canvas v1 testing extension) use. Python / Go / .NET / Java parity is planned as a follow-up once the wire contract has stabilized through real usage. I'll open a tracking issue for the parity work rather than expanding the scope of this PR. |
4 tasks
Contributor
Author
|
Filed tracking issue #1373 for Python / Go / .NET / Java parity. |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
The docstring on SessionOptions.canvases referenced [`CanvasDeclaration`] without a path. Since CanvasDeclaration lives in crate::canvas, rustdoc could not resolve the link and cargo doc failed under -D rustdoc::broken_intra_doc_links. Use the fully qualified path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Cross-SDK Consistency ReviewThis PR explicitly scopes canvas extensibility to Node.js and Rust, as stated in the PR description. That's a reasonable phased approach for a complex new feature surface. Status of other SDKs
New public API surface (Node.js) not yet in other SDKs
RecommendationConsider opening tracking issues for canvas support in Python, Go, .NET, and Java once the Node/Rust implementation stabilizes, to ensure feature parity across all SDK languages. No blocking changes needed for this PR given the intentional scoping.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds SDK support for canvas extensibility: agent-invokable canvas surfaces declared by the host or by 3rd-party extensions, with lifecycle events surfaced to the host.
What's added
Rust + Node — canvas declaration and dispatch
session.canvas.open,session.canvas.focus,session.canvas.close,session.canvas.reload,session.canvas.invokeAction).createCanvasNode factory and matching Rust types for declaring canvases.canvasesfield onSessionConfig/ResumeSessionConfigdeclares the host's canvases at session start and is preserved on the wire payload.requestCanvasRenderersession-level flag opts the session into canvas rendering.requestExtensionssession-level flag opts the session into the extension surface.instance_idis required onCanvasOpenContext— every canvas open is keyed by an agent-chosen stable identifier (Rust + Node).HostedExtensionsurface; host canvases register through the same path as 3rd-party extensions.Rust + Node — instance rehydrate on resume
CanvasInstanceRehydratetype (extensionId,canvasId,instanceId, optionalurl).ResumeSessionConfig.openCanvasInstances(Node) andopen_canvas_instancesfield +with_open_canvas_instancesbuilder (Rust) let the host re-attach known live canvas instances after a CLI restart, so action invocations against those instances succeed without requiring a re-open.Validation
cargo buildclean.npm run typecheckclean.