Repository navigation
test(mcp): resolve baseline toml path and restore mcp server integration tests (fixes #527) - #567
Open
i-am-paradox wants to merge 2 commits into
Open
i-am-paradox wants to merge 2 commits into
i-am-paradox wants to merge 2 commits into
Conversation
Marc-cn
requested changes
Oct 9, 2026
Marc-cn
left a comment
Collaborator
There was a problem hiding this comment.
Thanks, this works: on main these two files give 19 passed / 8 skipped, on your branch 28 passed / 0 skipped.
Before merging:
- AI disclosure. AI_POLICY.md asks for the tool(s) and which parts are AI-generated, plus an
Assisted-by: AGENT:MODELtrailer. You did both on #537; here the tool is not named and the commit has no trailer. Please add them. - The fallback path is dead code.
files("darnit_baseline")already resolves topackages/darnit-baseline/src/darnit_baseline/openssf-baseline.toml, the same file. Please drop the fallback in both files. test_audit_with_tags_filter: why hardcodeSTAGE1-REF-SECURITY-01? It hasdomain = "VM", so the filter is right and the ID check was the wrong proxy. #527 left this as an open decision. Can you explain your reasoning, and whether the test can assert on the control's domaininstead?
Optional: these tests went dark because a missing TOML is a skip. Making that a failure would stop it happening again.
Heads-up: #570 also touches tests/darnit/server/test_factory.py, so you may need to rebase.
i-am-paradox
force-pushed
the
fix/527-mcp-server-integration-tests
branch
from
October 9, 2026 22:50
7229549 to
9ee28bf
Compare
Contributor
Author
|
Addressed review points:
|
mlieberman85
requested changes
Oct 10, 2026
mlieberman85
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the quick turnaround. The path, marker and domain assertion all look right, and CI now runs all 8 MCP tests.
- The spawned
darnit servedoesn't get the test isolation.tests/conftest.pymonkeypatches the operator-config and data-root functions in-process, but the subprocess inheritsHOME. Sotest_get_pending_data_and_confirm_project_datawrites a realsecurity_contactconfirmation, under the local user's name, into the developer's own operator store, and every test reads the developer's~/.config/darnit/config.toml. Could you add one fixture that buildsenvfrommcp.client.stdio.get_default_environment()withHOME/XDG_CONFIG_HOME/XDG_DATA_HOMEpointed intotmp_path, and use it for everyStdioServerParameters? LaunchingPath(sys.executable).with_name("darnit")directly avoidsuv runneeding a warm cache under the new HOME. I checked that this keeps the confirmation in tmp. - Please make the round trip check the effect: pass
owner/repotoget_pending_data, and assertsecurity_contactis asked for beforeconfirm_project_dataand gone after it. test_audit_with_tags_filter: addassert resultsso an empty result can't pass vacuously.test_factory.py::test_openssf_baseline_toml: drop the remainingexcept ImportError: pytest.skip(...)so a broken import fails the way a missing TOML does, and move theimportlib.resourcesimport to the top of the module.
Drafted with Claude Code; reviewed and posted by me.
…ion tests (fixes darnitdevorg#527) - Resolve openssf-baseline.toml via importlib.resources.files without dead-code fallbacks - Assert baseline TOML file presence so missing configs fail tests instead of skipping silently - Mark test_mcp_server.py with pytest.mark.integration so CI integration test suite runs the module - Update expected tools count to 22 in test_server_starts_and_lists_tools (features 025 and 041) - Assert control domain directly in test_audit_with_tags_filter against baseline controls table - Add test_get_pending_data_and_confirm_project_data round trip Fixes darnitdevorg#527 Assisted-by: Antigravity:eni-gemini-2.5-pro Signed-off-by: Paradox <159635503+i-am-paradox@users.noreply.github.com>
- Provide mcp_server_params fixture with HOME and XDG pointed into tmp_path - Launch darnit binary directly to avoid uv cache requirements under isolated HOME - Symlink existing tree-sitter language pack cache to prevent network fetches in tests - Isolate test_repo directory and assert security_contact presence and removal in round trip - Add non-empty assertion on results in test_audit_with_tags_filter - Move importlib.resources.files to top of test_factory.py and drop ImportError skip Assisted-by: Antigravity:eni-gemini-2.5-pro Signed-off-by: Paradox <159635503+i-am-paradox@users.noreply.github.com>
i-am-paradox
force-pushed
the
fix/527-mcp-server-integration-tests
branch
from
October 10, 2026 22:51
9ee28bf to
a939fce
Compare
Contributor
Author
|
Addressed the review feedback in a939fce:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #527. Restores skipped MCP server integration tests and aligns assertions with current framework tools and configuration paths:
openssf-baseline.tomlviaimportlib.resources.files("darnit_baseline")directly without dead-code fallbacks intests/integration/test_mcp_server.pyandtests/darnit/server/test_factory.py.test_mcp_server.py) so missing configurations fail tests immediately rather than skipping silently.tests/integration/test_mcp_server.pywithpytest.mark.integrationso CI (pytest tests/ -m integration) discovers and executes the test module.expected_toolsintest_server_starts_and_lists_toolsfrom 18 to 22, including harness-loop (run_next_action,submit_action_result) and judgment (submit_judgment,confirm_pass_candidate) tools.test_audit_with_tags_filterto assert on each returned control's actual domain definition againstopenssf-baseline.tomlcontrols table rather than relying on control ID substring matching.test_get_pending_data_and_confirm_project_datatestingget_pending_dataandconfirm_project_dataround trips over MCP.Type of Change
Testing
uv run pytest tests/ -v)uv run ruff check .)Ran locally:
uv run pytest tests/integration/test_mcp_server.py tests/darnit/server/test_factory.py -v: 28 passeduv run ruff check tests/integration/test_mcp_server.py tests/darnit/server/test_factory.py: 0 errorsuv run ruff format --check tests/integration/test_mcp_server.py tests/darnit/server/test_factory.py: cleanuv run python scripts/validate_sync.py --verbose: all validations successfulAI assistance
Assisted by Antigravity (Gemini 2.5 Pro) for tracing test skips and drafting test assertions. All changes reviewed, verified, and tested against local pytest suite and sync validator.
Additional Notes
Fixes #527