Skip to content

Refactor scip-ruby repo tests and add packaging coverage#252

Draft
jasonhawkharris wants to merge 4 commits into
scip-ruby/masterfrom
jhh/re-write-test-suite
Draft

Refactor scip-ruby repo tests and add packaging coverage#252
jasonhawkharris wants to merge 4 commits into
scip-ruby/masterfrom
jhh/re-write-test-suite

Conversation

@jasonhawkharris

@jasonhawkharris jasonhawkharris commented Jun 10, 2026

Copy link
Copy Markdown

Refactors the end-to-end test coverage for scip-ruby by separating repo indexing tests from gem packaging tests.

Repo tests now run the freshly built //main:scip-ruby binary directly against pinned OSS Ruby projects and assert that indexing produces a non-empty SCIP index. This removes the previous Bundler/gem-installation setup, shared bundle cache, repo patches, and disabled CI comments.

This also adds a dedicated packaging test suite that builds a current-platform scip-ruby gem, installs it into a temporary Ruby project through Bundler, runs the installed scip-ruby executable, and verifies that it produces a non-empty SCIP index. The gem build rules now support current-platform test gems.

Docs and CI/Buildkite configuration were updated so repo tests and packaging tests run as separate jobs with clearer ownership and failure modes.

Motivation

The previous repo test setup tried to validate too many things at once: real-project indexing, gem packaging, Bundler installation, Ruby toolchain mutation, dependency caching, and local patching of third-party repos.

That made the tests fragile, hard to run in CI and difficult to debug. A failure could come from the indexer itself, the packaging path, Bundler behavior, cache synchronization, platform-specific patches, or the Ruby environment itself.

Splitting the coverage gives us clearer confidence:

  • repo tests verify that the current indexer can process real Ruby projects,
  • packaging tests verify that the built gem can be installed and run like users install it,
  • CI can run both paths without the old shared bundle-cache workflow.

Test plan

See included automated tests:

  • ./bazel test //test/scip/repos --config=dbg
  • ./bazel test //test/scip/packaging --config=dbg

CI and Buildkite now run the repo and packaging test suites separately.

@jasonhawkharris jasonhawkharris force-pushed the jhh/re-write-test-suite branch from c42d303 to 9a2c415 Compare June 11, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant