optimize: Tweak Integration Tests Flow for Speed + Cost by kalverra · Pull Request #23408 · smartcontractkit/chainlink

kalverra · 2026-08-13T20:51:35Z

Intent

Squeeze out a little more Integration Tests speed.

Big Changes

Replace `wasmtest`

Replace the old wasmtest package with new wasmbuild from chainlink-common. Also replaces any remaining go build calls in code to ad-hoc build wasm binaries.

Gives us better performance and smarter caching, while distributing the process to the rest of the company (like cre-cli).

Small Changes

Replaced smartcontractkit/.github/actions/ctf-setup-go with actions/setup-go@v7 across workflow files. Drops wrapper action execution overhead and eliminates unused cache restore steps for precompiled test binaries.
Switched to FastScryptParams in system-test crypto key generation and enabled InsecureFastScrypt: true in base node configs. Speeds up node startup and reduces CPU/memory usage during local key generation.
Replaced fixed time.Sleep with JSON-RPC polling against the vault gateway in updateVaultCapabilityConfigInRegistry. Prevents unnecessary static wait times while verifying on-chain config propagation.
Filtered unit-style tests from the CRE smoke test runner grep in CI workflows. Prevents executing redundant non-E2E tests in the integration test suite.
Removed deprecated enforce-ctf-version check from integration test workflows. Cleans up legacy CI gates.

github-actions · 2026-08-13T20:52:49Z

✅ No conflicts with other open PRs targeting develop

trunk-io · 2026-08-13T21:16:03Z

_{View Full Report ↗︎ ⋅ Docs}

github-actions · 2026-08-14T01:12:03Z

I see you updated files related to core. Please run make gocs in the root directory to add a changeset as well as in the text include at least one of the following tags:

#added For any new functionality added.
#breaking_change For any functionality that requires manual action for the node to boot.
#bugfix For bug fixes.
#changed For any change to the existing functionality.
#db_update For any feature that introduces updates to database schema.
#deprecation_notice For any upcoming deprecation functionality.
#internal For changesets that need to be excluded from the final changelog.
#nops For any feature that is NOP facing and needs to be in the official Release Notes for the release.
#removed For any functionality/config that is removed.
#updated For any functionality that is updated.
#wip For any change that is not ready yet and external communication about it should be held off till it is feature complete.

Copilot

Pull request overview

Risk Rating: MEDIUM — Changes are primarily test/CI infrastructure, but they affect critical integration workflows, caching behavior, and WASM build mechanics; regressions here can block merges and inflate CI cost.

This PR focuses on speeding up and simplifying the integration/system test pipeline (runner usage, caching, and test setup), while also making system tests cheaper by using faster scrypt settings and consolidating matrix prep/setup.

Changes:

Switch Go WASM compilation in tests/system-tests to chainlink-common/pkg/wasmbuild and add/extend .wasm-cache caching in GitHub Actions.
Update system-test crypto key encryption to use FastScryptParams and enable InsecureFastScrypt in generated node configs (test-only).
Refactor integration workflows: remove enforce-ctf-version, compute CRE matrix in setup, and replace custom ctf-setup-go with actions/setup-go.

Scrupulous human review recommended (high-impact areas):

.github/workflows/integration-tests.yml (matrix generation, cache keys, and job dependencies/conditions)
system-tests/tests/smoke/cre/vault_don_test_helpers.go (new propagation wait semantics)
core/internal/testutils/wasmtest/wasm.go (test build/caching behavior now delegated to wasmbuild)

Reviewed changes

Copilot reviewed 68 out of 76 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
system-tests/tests/smoke/cre/vault_don_test_helpers.go	Registry update wait now polls gateway
system-tests/tests/go.mod	Bump chainlink-common & brotli
system-tests/tests/go.sum	go.sum updates for bumps
system-tests/lib/go.mod	Bump chainlink-common & brotli
system-tests/lib/go.sum	go.sum updates for bumps
system-tests/lib/crypto/stellar.go	Use `FastScryptParams`
system-tests/lib/crypto/sol.go	Use `FastScryptParams`
system-tests/lib/crypto/p2p.go	Use `FastScryptParams`
system-tests/lib/crypto/dkgrecipient.go	Use `FastScryptParams`
system-tests/lib/crypto/aptos.go	Use `FastScryptParams`
system-tests/lib/crypto/evm.go	Import formatting cleanup
system-tests/lib/cre/workflow/workflow.go	Import formatting cleanup
system-tests/lib/cre/workflow/secrets.go	Import formatting cleanup
system-tests/lib/cre/workflow/registry.go	Import formatting cleanup
system-tests/lib/cre/workflow/compile.go	Use `wasmbuild.Compile` for Go workflows
system-tests/lib/cre/types.go	Import ordering cleanup
system-tests/lib/cre/sharding/sharding.go	Import formatting cleanup
system-tests/lib/cre/grpc_source_mock/server.go	Import ordering cleanup
system-tests/lib/cre/grpc_source_mock/auth.go	Update JWT authenticator construction
system-tests/lib/cre/grpc_source_mock/auth_test.go	Add interceptor test
system-tests/lib/cre/features/vault/vault.go	Import ordering cleanup
system-tests/lib/cre/features/stellar/stellar.go	Import formatting cleanup
system-tests/lib/cre/features/stellar/stellar_read_fixture.go	Import formatting cleanup
system-tests/lib/cre/features/stellar/stellar_forwarder.go	Import formatting cleanup
system-tests/lib/cre/features/stellar/stellar_config.go	Import formatting cleanup
system-tests/lib/cre/features/solana/v2/solana.go	Import ordering cleanup
system-tests/lib/cre/features/http_trigger/http_trigger.go	Import formatting cleanup
system-tests/lib/cre/features/http_action/http_action.go	Import formatting cleanup
system-tests/lib/cre/features/evm/v2/evm.go	Import re-org / ordering
system-tests/lib/cre/features/don_time/don_time.go	Import formatting cleanup
system-tests/lib/cre/features/cron/cron.go	Import formatting cleanup
system-tests/lib/cre/features/consensus/v2/consensus.go	Import ordering cleanup
system-tests/lib/cre/features/aptos/aptos.go	Import formatting cleanup
system-tests/lib/cre/features/aptos/aptos_workers.go	Import ordering cleanup
system-tests/lib/cre/features/aptos/aptos_helpers.go	Import formatting cleanup
system-tests/lib/cre/features/aptos/aptos_forwarder.go	Add depguard nolints on test-only imports
system-tests/lib/cre/environment/state.go	Import formatting cleanup
system-tests/lib/cre/environment/jobs.go	Import ordering cleanup
system-tests/lib/cre/environment/environment.go	Import formatting cleanup
system-tests/lib/cre/environment/dons.go	Import formatting cleanup
system-tests/lib/cre/environment/create_jd_jobs_op.go	Import formatting cleanup
system-tests/lib/cre/environment/config/config.go	Import formatting cleanup
system-tests/lib/cre/environment/config/config_test.go	Import formatting cleanup
system-tests/lib/cre/environment/blockchains/tron/tron.go	Import formatting cleanup
system-tests/lib/cre/environment/blockchains/stellar/stellar.go	Import formatting cleanup
system-tests/lib/cre/environment/blockchains/solana/solana.go	Import ordering cleanup
system-tests/lib/cre/environment/blockchains/blockchains.go	Import formatting cleanup
system-tests/lib/cre/don/jobs/jobs.go	Import formatting cleanup
system-tests/lib/cre/don/config/config.go	Enable `InsecureFastScrypt` in core config
system-tests/lib/cre/don.go	Import ordering cleanup
system-tests/lib/cre/contracts/registry_pickup_wait.go	Import formatting cleanup
system-tests/lib/cre/contracts/keystone_test.go	Import ordering cleanup
system-tests/lib/cre/contracts/contracts.go	Import ordering cleanup
integration-tests/load/go.mod	Bump chainlink-common & brotli
integration-tests/load/go.sum	go.sum updates for bumps
integration-tests/go.mod	Bump chainlink-common & brotli
integration-tests/go.sum	go.sum updates for bumps
go.mod	Bump chainlink-common & brotli
go.sum	go.sum updates for bumps
devenv/go.mod	Bump geth/chainlink deps; add indirects
devenv/go.sum	go.sum updates for bumps
deployment/go.mod	Bump chainlink-common & brotli
deployment/go.sum	go.sum updates for bumps
core/services/job/wasm_file_spec_factory_test.go	Build WASM using `wasmbuild.Compile`
core/scripts/go.mod	Bump chainlink-common & brotli
core/scripts/go.sum	go.sum updates for bumps
core/internal/testutils/wasmtest/wasm.go	Replace bespoke cache/build logic with `wasmbuild`
core/internal/testutils/wasmtest/wasm_test.go	Simplify wasmtest coverage to smoke-check output
.github/workflows/integration-tests.yml	Restructure setup/matrix, setup-go, and WASM cache
.github/workflows/integration-in-memory-tests.yml	Replace `ctf-setup-go` with `actions/setup-go`
.github/workflows/cre-system-tests.yaml	Allow precomputed matrix; switch setup-go
.github/workflows/cre-regression-system-tests.yaml	Switch setup-go; GITHUB_OUTPUT write style
.github/workflows/cre-mixed-env-tests.yaml	Switch setup-go; YAML description formatting
.github/workflows/ci-core.yml	Extend WASM cache key inputs; whitespace cleanup
.github/workflows/ccip-system-tests.yaml	Switch setup-go usage
.github/AGENTS.md	Add guidance for writing to GITHUB_OUTPUT

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

github-actions · 2026-08-14T15:52:44Z

CORA - Pending Reviewers

Codeowners Entry	Overall	Num Files	Owners
`*`	💬	54	@smartcontractkit/foundations, @smartcontractkit/core
`/core/services/job/`	💬	1	@smartcontractkit/foundations, @smartcontractkit/core
`/core/services/workflows/`	💬	5	@smartcontractkit/keystone
`/.github/**`	💬	8	@smartcontractkit/devex-cicd, @smartcontractkit/devex-tooling, @smartcontractkit/core
`go.mod`	💬	7	@smartcontractkit/core, @smartcontractkit/foundations
`go.sum`	💬	7	@smartcontractkit/core, @smartcontractkit/foundations
`integration-tests/go.mod`	💬	1	@smartcontractkit/core, @smartcontractkit/devex-tooling, @smartcontractkit/foundations
`integration-tests/go.sum`	💬	1	@smartcontractkit/core, @smartcontractkit/devex-tooling, @smartcontractkit/foundations

For more details, see the full review summary.

cl-sonarqube-production · 2026-08-14T17:27:57Z

Quality Gate failed

Failed conditions
2 New Blocker Issues (required ≤ 0)
C Reliability Rating on New Code (required ≥ A)
E Security Rating on New Code (required ≥ A)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Adds custom lint rules in tools/gorules/ to prevent specific bad patterns that standard golangci couldn't handle alone.

Replaces old wasmtest package with new, universal wasmbuild for better performance and caching

kalverra mentioned this pull request Aug 13, 2026

optimize: integration test speedups #23399

Closed

kalverra force-pushed the DX-4988/integration-test-speedups branch 2 times, most recently from fad0539 to 0484b20 Compare August 14, 2026 13:31

kalverra marked this pull request as ready for review August 14, 2026 14:22

kalverra requested review from a team as code owners August 14, 2026 14:22

kalverra requested a lite review from Copilot August 14, 2026 14:22

product-security-plaid-production Bot requested review from Tofel, jmank88, patrickhuie19 and skudasov August 14, 2026 14:22

Copilot started reviewing on behalf of kalverra August 14, 2026 14:23 View session

Copilot AI reviewed Aug 14, 2026

View reviewed changes

Comment thread system-tests/tests/smoke/cre/vault_don_test_helpers.go

Comment thread core/internal/testutils/wasmtest/wasm.go Outdated

Comment thread core/internal/testutils/wasmtest/wasm_test.go Outdated

jmank88 reviewed Aug 14, 2026

View reviewed changes

Comment thread devenv/go.mod Outdated

kalverra requested a review from a team as a code owner August 14, 2026 15:52

kalverra added 8 commits August 14, 2026 13:30

optimize: integration test speedups

185adde

fix matrix

46da754

fix: go setup

e3a50ff

revert: regression bucket changes

c119416

optimize: small speedups

232f0d3

optimize: disk + IO focused runners

21f9d5d

optimize: simplify go setup

a464dbd

cleanup

0820c10

kalverra added 10 commits August 14, 2026 13:30

DEBUG: trigger new run

354268c

optimize: wasm builds and caches

672c017

debug: check cached

b34ee7b

lint

d097d29

lint

65c4bce

feat: custom lint rules

7bcf9a7

Adds custom lint rules in tools/gorules/ to prevent specific bad patterns that standard golangci couldn't handle alone.

chore: replace old wasmtest

0cc0ecd

Replaces old wasmtest package with new, universal wasmbuild for better performance and caching

revert: remove custom lint rules

28dbf97

tidy

ad08207

lint

3a366f9

kalverra force-pushed the DX-4988/integration-test-speedups branch from 3074420 to 3a366f9 Compare August 14, 2026 17:31

Navigation Menu

Uh oh!

optimize: Tweak Integration Tests Flow for Speed + Cost - #23408

optimize: Tweak Integration Tests Flow for Speed + Cost#23408
kalverra wants to merge 18 commits into
developfrom
DX-4988/integration-test-speedups

kalverra commented Aug 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 13, 2026

Uh oh!

trunk-io Bot commented Aug 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 14, 2026 •

edited

Loading

Uh oh!

cl-sonarqube-production Bot commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kalverra commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Intent

Big Changes

Replace wasmtest

Small Changes

Uh oh!

github-actions Bot commented Aug 13, 2026

Uh oh!

trunk-io Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CORA - Pending Reviewers

Uh oh!

cl-sonarqube-production Bot commented Aug 14, 2026

Quality Gate failed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kalverra commented Aug 13, 2026 •

edited

Loading

Replace `wasmtest`

trunk-io Bot commented Aug 13, 2026 •

edited

Loading

github-actions Bot commented Aug 14, 2026 •

edited

Loading