Skip to content

feat(github): add typed Copilot and UI outputs - #3403

Open
SamMorrowDrums wants to merge 4 commits into
sammorrowdrums-typed-label-governance-toolsfrom
sammorrowdrums-typed-copilot-ui-outputs
Open

SamMorrowDrums wants to merge 4 commits into
sammorrowdrums-typed-label-governance-toolsfrom
sammorrowdrums-typed-copilot-ui-outputs

Conversation

@SamMorrowDrums

@SamMorrowDrums SamMorrowDrums commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Completes the typed-output stack with concrete Copilot and app-only ui_get schemas, letting modern MCP clients validate results without guessing from text. Bundled issue/PR Apps consume both modern envelopes and legacy responses, including reviewer avatars.

Why

Final layer of the native stack rooted at #3385, directly based on #3402. Registry audits find schemas for all 134 definitions, including duplicate registrations. Modern clients gain explicit output contracts while legacy clients retain successful handler text.
Fixes # — N/A; completes the stack rather than closing a separate issue.

What changed

  • Adds concrete Copilot assignment/intent/review outputs and all seven ui_get method payloads. Canonical snapshots replace duplicate typed-named snapshots.
  • Preserves input coercions/validation order, removes caller-controlled compatibility dispatch, and returns unknown-method errors without OAuth upgrades; genuine method scopes remain enforced.
  • Updates nine readers in three Apps via a shared dual-era parser, preserving avatars and nullable issue types. Adds native Node, schema, nonempty/null wire, and actual HTTP scope-middleware tests.
  • Restores nine registered input descriptions to exact immutable main 71ef8266e48110974b13aef50b4df6ff9914ff68 wording, preserving enums, types, defaults, bounds, and runtime behavior. Updates description assertions, six canonical snapshots, and generated docs.
  • Documents non-DTO PreserveHandlerContent exceptions. No foundation rebase or optional stateless-era performance patch is included.

MCP impact

  • No tool or API changes
  • Tool schema or behavior changed — supported modern 2026-07-28 receives outputSchema/typed structuredContent; ordinary declared success JSON text equals its DTO. 2025-11-25, empty/stateless, and literal unknown fixtures receive neither and retain legacy success text.
  • New tool added

Modern ui_get is method-tagged rather than flat; Apps support both. Modern review text is {"status":"requested"}, versus empty legacy text. Some DTO fields intentionally differ from legacy: this is not a blanket unchanged-shape claim.

Exact registered-input parity on final tree: independent audit against immutable main 71ef8266 covers 2,984 configurations / 226,852 comparisons, with zero differences, including descriptions. This is exhaustive all-variant input parity, not just default-catalog parity. Evidence: coordinator artifacts files/input-audit-description-fix/summary.json. All input constraints and runtime behavior remain unchanged.

Accepted stack error change: issue_read/get 404 changes from main's JSON-RPC error (code 0) to isError: true with the same message in both eras. Historical Pi/Codex/Inspector checks confirm model-visible execution errors in the protocol's tool-error form; the reverse would be a regression.

Prompts tested (tool changes only)

  • Automated equivalents of “Assign Copilot to issue 7,” “Suggest or directly assign Copilot with rationale/confidence,” and “Request review for PR 7,” across four protocol fixtures and input/error branches.
  • “Load labels, assignees, milestones, issue types, branches, issue fields, and reviewers.” All methods, nonempty/null outputs, DTO-text equality, both eras, avatars, and malformed envelopes are covered.
  • Historical exact-935b77a6 Inspector UI evidence: seven methods × main legacy/top legacy/top modern = 21 successes; AJV 7/7 and legacy content parity 7/7. Production parser consumed actual wire outputs, all nine reader integrations were asserted, and 1000-assignee/1000-reviewer avatars passed. Six unknown-method cases made zero measured API calls. Responses are deterministic provider fixtures over actual MCP transports, not live GitHub execution.
  • Final exact-255c31f6 harness reran list-only discovery for 91 Inspector tools: legacy 259,341 B, modern 413,154 B, byte-identical to 935b77a6 captures in both eras. Nine corrected descriptions are outside this default catalog. Captures: rounds/list-only-255c31f6/{measurements.json,list-captures}. Provider, cold-memory, UI/AJV, and independent full-suite harness were not rerun on final head.

Security / limits

  • No security or limits impact
  • Auth / permissions considered — app-only visibility retained; historical 935b77a6 actual HTTP tests prove compatibility keys cannot redirect dispatch, invalid methods do not challenge, and valid methods retain repo/read:org checks.
  • Data exposure, filtering, or token/size limits considered — DTOs constrain fields; avatars remain available. Discovery/memory costs are explicitly accepted below.

Accepted payload-cap exception: final 91-tool compact discovery is 413,154 B versus main's 259,341 B (1.593092×), exceeding the self-imposed 1.5× cap (389,011.5 B). This is an accepted exception, not a cap PASS: output schemas are the feature and legacy tool-success text remains unchanged. Restored avatar schema added 88 B to the earlier 413,066 B measurement.

Historical 130-tool real-library HTTP fixture (935b77a6, not remeasured on 255c31f6): main legacy/modern 372,644/374,890 B and top legacy/modern 373,003/546,247 B. Top legacy is byte-identical to prior 2c6d0826, not to main's catalog. That follow-up changed only two ui_get avatar string properties and required entries; all other tool objects/input schemas were unchanged. SHA256: legacy 3ddac9b5a325cac3f5cd7591e2b4dfcd308fdeca72e56de31cd6e97f76146d2e, modern 056f68b5c61f277b9c1c0e7894797a9d0fb6a86566d8df22a017209b2f3d0c82.

Historical warm performance (935b77a6, 8 × 25 iterations, original SDK, no overlays): versus immutable main, registration improves 26.88%, legacy HTTP latency improves 11.46%, and modern latency is statistically unchanged (p=.959)—not faster. Modern B/op/allocations increase 13.05%/13.82%; legacy B/op decreases 1.82% while allocations increase 11.88%. Versus 2c6d0826, HTTP timing/B/op/allocations do not regress; registration improves 3.71%. Cache pointers/backing slices remain stable: 203 initial misses, then zero new misses and 266 hits/request.

Measurements use NewHTTPMcpHandler with mock scopes/providers, not live authenticated request latency. Historical command: GOPROXY=off go test -mod=mod -modfile=<pinned-arm>.mod -run '^$' -bench '^(BenchmarkLibraryHTTPList|BenchmarkWarmRegistrationProcessSchemaCache)$' -count=8 -benchtime=25x -benchmem, then benchstat — performance GO under accepted tradeoffs. TestFinal(AdvertisedSchemas|WireCapture) and diagnostic TestLibraryHTTPPointerIdentity — PASS on 935b77a6; diagnostic SDK copy was not used for timing acceptance. Toolchain: Go 1.27.1 linux/amd64, Intel Core Ultra 9 185H, SDK 1.8.0/jsonschema-go 0.4.3.

Accepted cold/retained-memory exception: historical 2c6d0826 once-per-process definitions+registration 20.207→158.076 ms and approximately +7.22 MiB post-GC global schema retention. These exclude OS launch/package initialization and were not rerun or eliminated on either follow-up.

Accepted provider scope/coverage limits: supported GitHub Copilot provider evidence is sufficient for this work; no additional direct-provider validation is required. Historical exact-2c6d0826 supported GitHub Copilot adapters passed input-only checks. Standalone OpenAI/Anthropic/PTC and direct provider acceptance of output schemas were not tested because credentials/supported paths were unavailable; explicitly accepted untested scope, not a compatibility claim. Direct credential-backed GitHub execution remains unverified. Hosted builds are separate from local/independent checks.

Inspector warning triage (historical 935b77a6 modern 91-tool captures): 734 warnings = 729 type-union warnings (728 output / 1 input) + 5 untyped-schema warnings (3 output / 2 input); no style/description categories. Examples: actions_get output properties.workflow type [null, object] is legal nullable JSON Schema with a dialect-portability warning; projects_get output properties.item.properties.fields.items.properties.value permits arbitrary JSON. All 91 captured output schemas compile with AJV2020 + formats: zero invalid schemas and zero unresolved references. Separately, actual UI outputs validate 7/7; this is not execution coverage for all 91 tools. Strict validation exits 0 with zero errors. Warnings are triaged as nonblocking, not claimed absent. Final-head list-only captures preserve the same default 91-tool catalog; exhaustive input parity is the separate audit above.

Tool renaming

  • I am renaming tools as part of this PR (e.g. a part of a consolidation effort)
    • I have added the new tool aliases in deprecated_tool_aliases.go
  • I am not renaming tools as part of this PR — names/stack membership unchanged.

Note: if you're renaming tools, you must add the tool aliases. For more information on how to do so, please refer to the official docs.

Lint & tests

  • Linted locally with ./script/lint — PASS, 0 issues on final tree.
  • Tested locally with ./script/test — PASS, full race suite on final tree (pkg/github 317.569s).

Final owner validation applies exactly to tree 0383eef64b606b57354c7c896e4f44b2946b5bc7:

  1. UPDATE_TOOLSNAPS=true go test ./pkg/github -run 'Test(GranularToolSnaps|GranularPullRequest|TypedGranularPullRequest|_FindDuplicate)' -count=1 — PASS; refreshed snapshots.
  2. go test ./pkg/github -run 'Test(TypedInputDescriptionsMatchMain|GranularToolSnaps|GranularPullRequest|TypedGranularPullRequest|_FindDuplicate)' -count=1 — PASS.
  3. script/lint — PASS, 0 issues.
  4. script/test — PASS, full race (pkg/github 317.569s).
  5. script/generate-docs — PASS.
  6. git diff --check and git diff --cached --check — PASS; published worktree clean.

Current-head gh pr checks 3403 --repo github/github-mcp-server --watch --interval 30 — PASS, exit 0 for hosted workflows. Fresh gh pr checks 3403 --repo github/github-mcp-server — PASS, exit 0 on 255c31f6; build, lint, docs, licenses, MCP diff/HTTP, and CodeQL checks pass. Subsequent REST read reported 21 checks: 20 completed successfully and a newly requested Copilot review in progress; no final-head review conclusion is claimed.

Historical 935b77a6 owner checks: UPDATE_TOOLSNAPS=true GOTMPDIR=/dev/shm/copilot-mcp-race-485bd8e4 go test ./... — PASS (pkg/github 38.177s); ordered script/lint — PASS (0 issues), GOTMPDIR=/dev/shm/copilot-mcp-race-485bd8e4 script/test — PASS (full race), script/generate-docs — PASS, git diff --check — PASS. Earlier owner full-race run pkg/github 318.432s is historical, not the final-tree run.

Historical 935b77a6 targeted/UI checks: UPDATE_TOOLSNAPS=true go test ./pkg/github -run 'TestTypedCopilot|TestTypedUIGet|TestUIGet' -count=1 — PASS (0.246s); cd ui && npm test — PASS (11/11); npm run typecheck — PASS; npm run build — PASS (four Apps). Registry flags-on, flags-off, all-definitions, each-definition — PASS, 134/134 output schemas.

Historical exact-935b77a6 independent harness: go test -race ./pkg/github -run 'Test(TypedCopilotAndUIWireOutputs|TypedUIGet.*|TypedCopilotUIErrorsPreserveLegacyText|TypedCopilotOutputSchemas|ToolDefinitionsUseConcreteOutputTypes)$' -count=1 — PASS (3.493s). GOFLAGS=-p=1 script/lint, script/test (full race), script/generate-docs, git diff --check, and clean-tree checks — PASS, exit 0. Independent npm test (11/11), npm run typecheck, npm run build — PASS after npm ci --ignore-scripts --no-audit --no-fund. UI runs used Node 22.23.3/npm 10.9.9; package Node ^26 engine warning disclosed. These independent full-suite/UI checks were not rerun on 255c31f6.

Historical Copilot review on exact 935b77a6 reported no findings; all four prior findings have proof-based replies and are resolved. Fresh review was requested via REST on 255c31f6 and its check was observed in progress; no result yet claimed.

Docs

  • Not needed
  • Updated (README / docs / examples) — clarified docs/typed-tool-schemas.md; regenerated documentation/canonical snapshots, including nine description corrections.

Final signed head 255c31f624d8519cd552716ea1c6bd41a7453582, tree 0383eef64b606b57354c7c896e4f44b2946b5bc7, direct base 15e359be93f75b4491f07fba7759bc61474862b4. GitHub REST signature verification is verified: true, reason valid; head equals live branch ref. Historical pre-correction head was 935b77a6dc7a9eee95ce20a2776b9737322251fd, tree 8ad8199f787238e3a2daad1a0f159aa3c7cef8f0. This body update changes neither head, native stack metadata, nor lifecycle; no merge is performed.

@SamMorrowDrums
SamMorrowDrums added this pull request to stack #3385 October 2, 2026 21:43
@SamMorrowDrums
SamMorrowDrums marked this pull request as ready for review October 5, 2026 10:26
Copilot AI balanced review requested due to automatic review settings October 5, 2026 10:26
@SamMorrowDrums
SamMorrowDrums requested a review from a team as a code owner October 5, 2026 10:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Caller-controlled compatibility metadata can redirect UI dispatch without the corresponding OAuth scope challenge.

Review effort: Balanced
Findings: 1 High severity

Open (1)
What changed in this PR

Extends the GitHub MCP Server’s typed-output support to Copilot tools and ui_get, targeting modern clients while retaining legacy text responses.

Changes:

  • Adds concrete output types, schemas, and compatibility input normalizers.
  • Adds protocol compatibility tests and typed tool snapshots.
File Description
pkg/​github/​ui_tools.go Returns typed UI results.
pkg/​github/​ui_tools_test.go Uses the typed UI snapshot.
pkg/​github/​typed_copilot_ui_outputs.go Defines output types, schemas, and normalizers.
pkg/​github/​typed_copilot_ui_outputs_test.go Tests wire outputs, errors, and schemas.
pkg/​github/​copilot.go Returns typed assignment and review results.
pkg/​github/​copilot_test.go Uses typed Copilot snapshots.
pkg/​github/​__toolsnaps__/​ui_get_typed.snap Captures UI output variants.
pkg/​github/​__toolsnaps__/​request_copilot_review_typed.snap Captures the null review output schema.
pkg/​github/​__toolsnaps__/​assign_copilot_to_issue_with_intent_typed.snap Captures intent-aware assignment output.
pkg/​github/​__toolsnaps__/​assign_copilot_to_issue_typed.snap Captures assignment output.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pkg/github/typed_copilot_ui_outputs.go
SamMorrowDrums and others added 2 commits October 5, 2026 22:58
Add protocol-gated output schemas and structured output DTOs for Copilot assignment, review requests, and UI data. Preserve legacy text and validate compatibility normalizers across supported protocol versions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@SamMorrowDrums
SamMorrowDrums force-pushed the sammorrowdrums-typed-copilot-ui-outputs branch from 647c049 to 2c6d082 Compare October 5, 2026 21:36
@SamMorrowDrums
SamMorrowDrums requested a balanced review from Copilot October 5, 2026 21:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Modern ui_get responses are incompatible with the bundled Apps’ existing response readers.

Review effort: Balanced
Findings: 2 High severity · 2 Medium severity

Open (4)

Comment thread pkg/github/typed_copilot_ui_outputs.go
Comment thread pkg/github/typed_copilot_ui_outputs.go Outdated
Comment thread pkg/github/typed_copilot_ui_outputs.go Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No blocking issues remain, and focused regression coverage addresses protocol compatibility, scope handling, and App decoding.

Review effort: Balanced
Findings: None

Resolved since last review (4)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Compatibility of the new UI test command with the declared Node 26 runtime remains unverified.

Review effort: Balanced
Findings: None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants