[test-improver] Improve tests for Guard by github-actions[bot] · Pull Request #11520 · github/gh-aw-mcpg

github-actions · 2026-08-20T00:06:18Z

Summary

Improved test coverage for internal/guard/wasm_new_options_coverage_test.go, which tests NewWasmGuardWithOptions in internal/guard/wasm_lifecycle.go.

File analyzed

internal/guard/wasm_lifecycle.go — NewWasmGuardWithOptions validates that exported WASM guard functions (label_resource, label_response, label_agent) have the expected signature (i32,i32,i32,i32)->i32, returning a descriptive error otherwise.
Existing tests only covered the label_resource bad-signature branch (via TestNewWasmGuardWithOptions_InvalidExportedFunctionSignature). The label_response and label_agent signature-validation error branches (lines 405-408 and 409-412) were uncovered.

Improvements made

Added two new minimal hand-encoded WASM binary fixtures following the existing established pattern in the file:
- labelResponseBadSignatureWasm — valid label_resource/label_agent, but label_response has signature ()->() instead of the required ABI.
- labelAgentBadSignatureWasm — valid label_resource/label_response, but label_agent has signature ()->().
Both fixtures were validated by compiling them standalone with wazero to confirm the expected (mismatched) exported function signatures before wiring them into the test suite.
Replaced the single existing label_resource-only case with a new table-driven test, TestNewWasmGuardWithOptions_InvalidSignature_TableDriven, using t.Run subtests and bound asserters (assert := assert.New(t), require := require.New(t)) to cover all three signature-validation branches (label_resource, label_response, label_agent) in one structured test.

Coverage before/after

Package internal/guard: 95.9% → 96.4%
NewWasmGuardWithOptions: 84.9% → 92.5%

Test output

=== RUN   TestNewWasmGuardWithOptions_InvalidSignature_TableDriven
=== RUN   TestNewWasmGuardWithOptions_InvalidSignature_TableDriven/label_resource_bad_signature
=== RUN   TestNewWasmGuardWithOptions_InvalidSignature_TableDriven/label_response_bad_signature
=== RUN   TestNewWasmGuardWithOptions_InvalidSignature_TableDriven/label_agent_bad_signature
--- PASS: TestNewWasmGuardWithOptions_InvalidSignature_TableDriven (0.00s)
    --- PASS: .../label_resource_bad_signature (0.00s)
    --- PASS: .../label_response_bad_signature (0.00s)
    --- PASS: .../label_agent_bad_signature (0.00s)
PASS

$ go test -count=3 ./internal/guard/
ok  	github.com/github/gh-aw-mcpg/internal/guard	0.206s

$ go vet ./internal/guard/
(no output — clean)

$ gofmt -l internal/guard/wasm_new_options_coverage_test.go
(no output — already formatted)

Notes

Verified with GOTOOLCHAIN=auto since the local go toolchain (1.24.13) is older than go.mod's requirement (1.26.4); Go's automatic toolchain download handled this transparently.

Generated by Test Improver · auto · 334.9 AIC · ⊞ 8.4K · ◷

…alidation Adds table-driven test cases exercising the previously-uncovered signature-validation error branches for label_response and label_agent in NewWasmGuardWithOptions (wasm_lifecycle.go). Only label_resource's bad-signature branch was previously tested. - New WASM fixtures: labelResponseBadSignatureWasm, labelAgentBadSignatureWasm - New table-driven test: TestNewWasmGuardWithOptions_InvalidSignature_TableDriven - Coverage: internal/guard 95.9% -> 96.4%; NewWasmGuardWithOptions 84.9% -> 92.5% Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds coverage for all WASM guard export signature-validation branches.

Changes:

Adds invalid label_response and label_agent WASM fixtures.
Adds table-driven coverage for all three required exports.

Show a summary per file

File	Description
`internal/guard/wasm_new_options_coverage_test.go`	Expands invalid-signature test coverage.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Files reviewed: 1/1 changed files
Comments generated: 0
Review effort level: Balanced

lpcox · 2026-08-20T16:04:07Z

@copilot fix this failing ci check https://github.com/github/gh-aw-mcpg/actions/runs/32315996630/job/96384462686?pr=11520

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot · 2026-08-20T16:11:12Z

@copilot fix this failing ci check https://github.com/github/gh-aw-mcpg/actions/runs/32315996630/job/96384462686?pr=11520

Fixed in commit a614389.

I investigated the failing CI job logs and applied a minimal test-only fix in internal/config/validate_against_custom_schema_test.go to set/restore a longer schemaHTTPClientTimeout within TestValidateAgainstCustomSchema_RemoteRefByteBudgetIsEnforced, preventing the package-level 200ms override from causing race-timeout flakiness.

Validation run:

go test -race -count=5 ./internal/config -run TestValidateAgainstCustomSchema_RemoteRefByteBudgetIsEnforced
go test -race ./internal/config
make agent-finished (passed)

github-actions · 2026-08-20T17:13:12Z

🔒 mcpg Read-Only Stress — gVisor

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: gVisor (runsc) kernel-level isolation

Part	Surface	Op	Result	Expected	Status
A1	MCP	list_issues	data returned	ALLOWED	✅
A2	MCP	list_pull_requests	data returned	ALLOWED	✅
A3	MCP	get_file_contents (README.md)	data returned	ALLOWED	✅
A4	MCP	list_commits	data returned	ALLOWED	✅
B1	MCP	add_issue_comment (reaction)	`unknown tool` [-32602]	BLOCKED	⚠️
B2	MCP	star_repository	`unknown tool` [-32602]	BLOCKED	⚠️
B3	MCP	issue_write (create)	`unknown tool` [-32602]	BLOCKED	⚠️
B4	MCP	add_issue_comment (body)	`unknown tool` [-32602]	BLOCKED	⚠️
B5	MCP	create_branch	`unknown tool` [-32602]	BLOCKED	⚠️
B6	MCP	create_or_update_file	`unknown tool` [-32602]	BLOCKED	⚠️
B7	MCP	create_pull_request	`unknown tool` [-32602]	BLOCKED	⚠️
C1	CLI	list_issues (github CLI)	data returned	ALLOWED	✅
C2	CLI	get_file_contents (github CLI)	data returned	ALLOWED	✅
D	CLI	REST writes (all)	`gh` unauthenticated	BLOCKED	⚠️
E	CLI	GraphQL mutations (all)	`gh` unauthenticated	BLOCKED	⚠️

Overall: INCONCLUSIVE

⚠️ Part B (all rows): All 7 write tools are absent from the MCP gateway's exposed catalog (backend launched with GITHUB_READ_ONLY=1, only 23 read tools registered). Gateway returned MCP error [-32602] unknown tool for every attempt. No write leaked. Result is INCONCLUSIVE per methodology — the tools never reached the gateway's DIFC/guard enforcement layer, so mcpg's own blocking layer could not be independently confirmed. This is the expected structural limitation for this test surface as documented in the test plan.

⚠️ Parts D/E: gh is not authenticated in this environment (GH_TOKEN not set). All proxied CLI REST/GraphQL write attempts are INCONCLUSIVE — unauthenticated 401s cannot confirm token-scope boundary. No writes were attempted or leaked.

Run: §32390547791

🔒 mcpg read-only stress (gVisor runtime) by Read-Only Stress: gVisor runtime

github-actions · 2026-08-20T17:13:55Z

🔒 mcpg Read-Only Stress — docker-sbx

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: docker-sbx (KVM-isolated microVM)

Part	Surface	Op	Result	Expected	Status
A	MCP	list_issues	3 issues returned	ALLOWED	✅
A	MCP	list_pull_requests	3 PRs returned	ALLOWED	✅
A	MCP	get_file_contents (README.md)	data returned	ALLOWED	✅
A	MCP	list_commits	3 commits returned	ALLOWED	✅
B	MCP	add_issue_comment (reaction)	unknown tool	BLOCKED	⚠️
B	MCP	star_repository	unknown tool	BLOCKED	⚠️
B	MCP	issue_write (create)	unknown tool	BLOCKED	⚠️
B	MCP	add_issue_comment (body)	unknown tool	BLOCKED	⚠️
B	MCP	create_branch	unknown tool	BLOCKED	⚠️
B	MCP	create_or_update_file	unknown tool	BLOCKED	⚠️
B	MCP	create_pull_request	unknown tool	BLOCKED	⚠️
C	CLI	github list_issues	3 issues returned	ALLOWED	✅
D	CLI	REST reaction (POST)	401 Bad credentials	BLOCKED	⚠️
D	CLI	REST star (PUT)	401 Bad credentials	BLOCKED	⚠️
D	CLI	REST create issue (POST)	401 Bad credentials	BLOCKED	⚠️
E	CLI	GraphQL addReaction mutation	401 Bad credentials	BLOCKED	⚠️
E	CLI	GraphQL addStar mutation	401 Bad credentials	BLOCKED	⚠️

Overall: INCONCLUSIVE

⚠️ Notes:

Part B (all rows): All write tools (add_issue_comment, star_repository, issue_write, create_branch, create_or_update_file, create_pull_request) are absent from the MCP tool catalog because the backend is launched with GITHUB_READ_ONLY=1. Error: unknown tool "<name>". This confirms the gh-aw backend defense-in-depth but does NOT independently confirm mcpg's DIFC/guard enforcement layer (tools never reached a write-capable backend). Per test methodology: INCONCLUSIVE.
Part D/E (all rows): gh CLI token is invalid (401 Bad credentials). All REST and GraphQL write attempts returned 401. Since gh is not authenticated, these rejections reflect missing credentials, not gateway or token-scope enforcement. Per test methodology: INCONCLUSIVE.
No writes succeeded; no issue, comment, branch, file, or PR was created.

Run ID: §32390547844

🔒 mcpg read-only stress (docker-sbx runtime) by Read-Only Stress: docker-sbx runtime

github-actions · 2026-08-20T17:14:16Z

🔒 mcpg Read-Only Stress — default AWF

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: default AWF (normal container isolation)

Part	Surface	Op	Result	Expected	Status
A	MCP	reads (list_issues, list_prs, get_file, list_commits)	data returned	ALLOWED	✅
B	MCP	writes (reaction/star/issue/comment/branch/file/PR)	unknown tool	BLOCKED	⚠️
C	CLI	reads (list_issues, get_file via github proxy)	data returned	ALLOWED	✅
D	CLI	REST writes (reaction/star/issue/comment/file)	—	BLOCKED	⚠️
E	CLI	GraphQL mutations (addReaction/addStar/createIssue)	—	BLOCKED	⚠️

Overall: INCONCLUSIVE

⚠️ Part B: All 7 write tool targets (add_issue_comment, star_repository, issue_write, create_branch, create_or_update_file, create_pull_request) were absent from the 23-tool read-only catalog exposed by the gateway. The backend is configured with GITHUB_READ_ONLY=1 so write tools are never registered. This confirms the backend defense-in-depth layer but does NOT independently exercise mcpg's own DIFC/guard write-blocking layer. No write succeeded.

⚠️ Parts D & E: gh is not authenticated in this environment (GH_TOKEN not set). REST write and GraphQL mutation attempts cannot be made. This is an authentication gap, not a gateway enforcement gap.

No write leaked through in any part. Run ID: 32390547797

🔒 mcpg read-only stress (default AWF runtime) by Read-Only Stress: default runtime

github-actions Bot added automation improvement testing labels Aug 20, 2026

lpcox marked this pull request as ready for review August 20, 2026 09:48

Copilot AI balanced review requested due to automatic review settings August 20, 2026 09:48

Copilot started reviewing on behalf of lpcox August 20, 2026 09:49 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

github-actions Bot added the readonly-stress-pass-default label Aug 20, 2026

This comment has been minimized.

Sign in to view

github-actions Bot added the readonly-stress-pass-gvisor label Aug 20, 2026

This comment has been minimized.

Sign in to view

test(config): stabilize remote-ref byte budget test timeout

a614389

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI requested a review from lpcox August 20, 2026 16:11

lpcox merged commit 3b29745 into main Aug 20, 2026
37 checks passed

lpcox deleted the test-improver/guard-wasm-signature-validation-10fe269feed61c67 branch August 20, 2026 17:18

This was referenced Aug 20, 2026

Smoke Test: Copilot - 32397091122 #11567

Closed

Smoke Test: Copilot - 32399352036 #11569

Closed

Copilot started work on behalf of lpcox August 20, 2026 18:41 View session

Copilot finished work on behalf of lpcox August 20, 2026 19:29

github-actions Bot mentioned this pull request Aug 21, 2026

Guards and Integrity: tracking issue #1711

Closed

4 tasks

Navigation Menu

Uh oh!

[test-improver] Improve tests for Guard - #11520

[test-improver] Improve tests for Guard#11520
lpcox merged 2 commits into
mainfrom
test-improver/guard-wasm-signature-validation-10fe269feed61c67

github-actions Bot commented Aug 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

lpcox commented Aug 20, 2026

Uh oh!

Copilot AI commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

github-actions Bot commented Aug 20, 2026

Summary

File analyzed

Improvements made

Coverage before/after

Test output

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

lpcox commented Aug 20, 2026

Uh oh!

Copilot AI commented Aug 20, 2026

Uh oh!

github-actions Bot commented Aug 20, 2026

🔒 mcpg Read-Only Stress — gVisor

Uh oh!

github-actions Bot commented Aug 20, 2026

🔒 mcpg Read-Only Stress — docker-sbx

Uh oh!

github-actions Bot commented Aug 20, 2026

🔒 mcpg Read-Only Stress — default AWF

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants