🌱 Enable parallel e2e test execution for faster local dev feedback by pedjak · Pull Request #2785 · operator-framework/operator-controller

pedjak · 2026-06-25T15:59:14Z

Description

Enable running standard and experimental e2e tests simultaneously on separate KIND clusters via make -j2 test-e2e test-experimental-e2e, cutting local e2e wall-clock time by ~40%. This is especially valuable when working with AI coding agents that benefit from faster feedback loops.

Benchmark

Mode	Wall clock	Speedup
Sequential (`make test-e2e && make test-experimental-e2e`)	25m 25s	—
Parallel (`make -j2 test-e2e test-experimental-e2e`)	15m 54s	~40% faster

Measured on a dev workstation (Fedora, 16 cores, 64GB RAM).

Linux prerequisite

Running multiple KIND clusters simultaneously requires sufficient inotify instances:

sudo sysctl fs.inotify.max_user_instances=512

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

netlify · 2026-06-25T15:59:19Z

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	`dbd2804`
🔍 Latest deploy log	https://app.netlify.com/projects/olmv1/deploys/6a3eb5a5eaef49000804d441
😎 Deploy Preview	https://deploy-preview-2785--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

openshift-ci · 2026-06-25T15:59:27Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tmshort for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot

Pull request overview

Enables running standard and experimental e2e suites concurrently by generating per-variant Makefile target chains (separate KIND clusters, kubeconfigs, and Prometheus host ports) and wiring the summary reporter to use a per-run PROMETHEUS_URL.

Changes:

Refactors test-e2e / test-experimental-e2e into an e2e-target macro that emits uniquely named per-variant targets and uses per-cluster kubeconfig files.
Updates the e2e summary generator to read Prometheus address from PROMETHEUS_URL and skip summary generation when it’s unset.
Separates experimental Prometheus NodePort host mapping (30901) and documents/ignores the new .kubeconfig/ directory.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`test/internal/summary/summary.go`	Switches summary Prometheus endpoint to `PROMETHEUS_URL` and skips when unset.
`Makefile`	Generates per-variant e2e target chains, isolates kubeconfig, sets `PROMETHEUS_URL`, and adds parallel-run guidance.
`kind-config/kind-config-2node.yaml`	Changes experimental Prometheus hostPort mapping to 30901.
`AGENTS.md`	Documents `make -j2 test-e2e test-experimental-e2e` parallel workflow and Linux inotify prerequisite.
`.gitignore`	Ignores `.kubeconfig/` per-cluster kubeconfig outputs.

Comments suppressed due to low confidence (1)

test/internal/summary/summary.go:187

defaultPromUrl is read from the environment at package init time, so setting PROMETHEUS_URL later (e.g., in TestMain before calling PrintSummary, or from another caller) will not be observed and summary generation will be skipped unexpectedly. Read the env var inside PrintSummary instead.

	if defaultPromUrl == "" {
		fmt.Printf("PROMETHEUS_URL not set; skipping summary generation")
		return nil
	}

	client, err := api.NewClient(api.Config{
		Address: defaultPromUrl,
	})

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-06-25T16:26:17Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.42%. Comparing base (c66f3af) to head (dbd2804).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2785      +/-   ##
==========================================
+ Coverage   70.40%   70.42%   +0.02%     
==========================================
  Files         143      143              
  Lines       10617    10625       +8     
==========================================
+ Hits         7475     7483       +8     
  Misses       2580     2580              
  Partials      562      562

Flag	Coverage Δ
e2e	`35.11% <ø> (-0.06%)`	⬇️
experimental-e2e	`52.43% <ø> (+0.01%)`	⬆️
unit	`59.50% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

tmshort · 2026-06-26T14:58:51Z

+Both suites can run simultaneously on separate KIND clusters:
+
+```bash
+make -j2 test-e2e test-experimental-e2e


This might be a bit redundant, as test-experimental-e2e should also be running the same tests as test-e2e (basically running the test-e2e with new features)

Ah... but they're run on separate clusters, and it's not combined?

Ah... but they're run on separate clusters, and it's not combined?

They are on different clusters with differently deployed OLM (different feature gates). There fwe need two clusters.

tmshort · 2026-06-26T15:37:51Z

+.PHONY: kind-load-%
+kind-load-%: kind-cluster-%
+	$(KIND) load docker-image $(OPCON_IMG) --name operator-controller-$*
+	$(KIND) load docker-image $(CATD_IMG) --name operator-controller-$*


Could the existing kind-load be updated to reference kind-load-%, so we can eliminate drift?
Same with other targets.

Could the existing kind-load be updated to reference kind-load-%, so we can eliminate drift? Same with other targets.

They could and should, but I would leave that for follow up PR, to avoid widening the scope.

Copilot

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

rashmigottipati

Changes look good to me overall. Will add the labels once the other review has been addressed

Run standard and experimental e2e tests simultaneously on separate KIND clusters via `make -j2 test-e2e test-experimental-e2e`, cutting local e2e wall-clock time by ~37% (25m → 16m). This is especially valuable when working with AI coding agents that benefit from faster feedback loops. Key changes: - Use Make pattern rules (kind-cluster-%, kind-load-%, etc.) with the stem identifying the variant. Cluster names, kubeconfig paths, and coverage names are derived by convention from the stem. Only variant-specific deviations need explicit overrides. - Remove .NOTPARALLEL directive (no longer needed with unique targets) - Isolate per-cluster kubeconfig files under .kubeconfig/ - Use separate KIND cluster names and prometheus host ports (30900 vs 30901) to avoid resource conflicts - Make PROMETHEUS_URL configurable via env var in summary reporter; skip summary generation when unset - Conditionally serialize kind-deploy when both targets run together to avoid races on shared files Linux prerequisite for parallel runs: sudo sysctl fs.inotify.max_user_instances=512 Co-Authored-By: Claude <noreply@anthropic.com>

pedjak · 2026-06-26T17:25:53Z

Changes look good to me overall. Will add the labels once the other review has been addressed

ptal

Copilot AI review requested due to automatic review settings June 25, 2026 15:59

openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026

Copilot started reviewing on behalf of pedjak June 25, 2026 15:59 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread Makefile Outdated

pedjak marked this pull request as ready for review June 25, 2026 20:44

Copilot AI review requested due to automatic review settings June 25, 2026 20:44

openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026

openshift-ci Bot requested review from OchiengEd and rashmigottipati June 25, 2026 20:44

Copilot started reviewing on behalf of pedjak June 25, 2026 20:44 View session

pedjak requested review from dtfranz and perdasilva and removed request for OchiengEd June 25, 2026 20:45

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread Makefile Outdated

Comment thread test/internal/summary/summary.go Outdated

pedjak force-pushed the parallel-e2e branch from 719337a to c741dc9 Compare June 26, 2026 11:04

Copilot AI review requested due to automatic review settings June 26, 2026 11:04

Copilot started reviewing on behalf of pedjak June 26, 2026 11:04 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread Makefile

pedjak force-pushed the parallel-e2e branch from c741dc9 to 096d018 Compare June 26, 2026 11:24

tmshort reviewed Jun 26, 2026

View reviewed changes

Comment thread Makefile Outdated

tmshort reviewed Jun 26, 2026

View reviewed changes

Comment thread Makefile

tmshort reviewed Jun 26, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings June 26, 2026 15:46

pedjak force-pushed the parallel-e2e branch from 096d018 to 7baf7dd Compare June 26, 2026 15:46

Copilot started reviewing on behalf of pedjak June 26, 2026 15:46 View session

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread Makefile Outdated

rashmigottipati reviewed Jun 26, 2026

View reviewed changes

pedjak force-pushed the parallel-e2e branch from 7baf7dd to dbd2804 Compare June 26, 2026 17:23

Uh oh!

Conversation

pedjak commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Benchmark

Linux prerequisite

Reviewer Checklist

Uh oh!

netlify Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for olmv1 ready!

Uh oh!

openshift-ci Bot commented Jun 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

codecov Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

tmshort Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

tmshort Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

pedjak Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tmshort Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

pedjak Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

rashmigottipati left a comment

Choose a reason for hiding this comment

Uh oh!

pedjak commented Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pedjak commented Jun 25, 2026 •

edited

Loading

netlify Bot commented Jun 25, 2026 •

edited

Loading

codecov Bot commented Jun 25, 2026 •

edited

Loading