Build and test conda-recipe-cf for Intel Python CI by jharlow-intel · Pull Request #204 · IntelPython/mkl_random

jharlow-intel · 2026-10-23T20:23:40Z

Description

Adds what Intel Python CI (gh-ci) needs from mkl_random: a ci/ directory and a new workflow, .github/workflows/intel-ci.yml. The workflow builds conda-recipe-cf for every platform and Python version the recipe declares, and tests each package in a clean conda-forge environment with ci/test.py. Intel's Jenkins then waits for that run, tests each package again on its own machines, scans it, and for a release tag publishes the packages it tested. Jenkins isn't switched on for mkl_random yet, so until it is, this only adds a GitHub Actions workflow.

ci/matrix.py, ci/build.py, ci/hooks.py: the shared gh-ci scripts, standard library only. Here they look for conda-recipe-cf/ first and build Linux and Windows on GitHub-hosted runners. hooks.py installs the test requirements from ci/config.yml, the way Jenkins does.
ci/test.py: runs pytest --pyargs mkl_random against the installed package and writes a JUnit report.
ci/benchmark.py: because benchmarks/ exists, Jenkins runs it once per pull request or tag, with the newest regular Linux package, on a bare-metal benchmarking machine. It runs the asv suite against the installed package: one quick sample of each benchmark on a pull request, a full measurement on a tag. A failing benchmark fails it; nothing compares the timings with a baseline yet.
ci/config.yml: the workflow, the artifact names (mkl_random {platform} Python {python}, as conda-package-cf.yml names them) and the test requirements, pytest, and the benchmark requirements, psutil, with asv from PyPI, as conda-forge has no asv for Python 3.15 yet.
conda-recipe-cf/conda_build_config.yaml: now lists the Python versions, the ones conda-package-cf.yml builds today, for ci/matrix.py, and numpy: '2'. Without a NumPy variant, conda-build falls back to NumPy 1.26, which has no builds for Python 3.13 and newer. conda-package-cf.yml passes --python and --numpy itself, so it is unaffected.
ci/README.md and AGENTS.md notes.

A pull request builds the oldest and the newest regular Python versions on Linux and Windows; pushes to master and tags build them all.

Miniforge is pinned to 26.7.2-0. Miniforge 26.9.4-0, released on 22 October, makes setup-miniconda's conda init fail on Windows runners, and the existing conda-package*.yml workflows, which install latest, hit it too on this pull request.

Overlap with #202. It also sets numpy: '2' in conda_build_config.yaml and drops Python 3.10, so whichever lands second needs a small rebase, and if 3.10 goes it should leave the python: list here too. conda-package-cf.yml and intel-ci.yml now both build conda-recipe-cf; retiring one can be a follow-up.

Verification

Tests: this pull request's Intel Python CI run built Linux and Windows for Python 3.10 and 3.15, the oldest and the newest, and ran ci/test.py against each package: 380 passed on each platform and version.
Lint: pre-commit run on every changed file, all hooks passing, and zizmor on intel-ci.yml, no findings.
python ci/matrix.py lists 14 builds, Linux and Windows for 3.10 to 3.15 and 3.14t; with --event pull_request it lists 3.10 and 3.15 on both.
ci/config.yml loads with gh-ci's own parser.

Not verified

The builds a tag or a push to the default branch adds, Python 3.11 to 3.14 and 3.14t; they use the same jobs.
The Jenkins side, which isn't enabled for mkl_random yet, the benchmark cell included.
ci/benchmark.py against mkl_random's suite. The same script ran mkl_fft's.

antonwolfy · 2026-10-26T21:29:32Z

+
+concurrency:
+  group: intel-ci-${{ github.ref }}
+  cancel-in-progress: true


With cancel-in-progress: true keyed on github.ref, two quick merges cancel the first commit's whole run. Jenkins, which waits for that run, gets no packages for it.

Suggested change

cancel-in-progress: true

cancel-in-progress: ${{ github.event_name == 'pull_request' }}

Applied in 6006c16, so a push to master no longer stops the run before it. For what it is worth, Jenkins only waits for pull request and tag runs, and every tag is its own ref, so it never lost packages this way.

antonwolfy · 2026-10-26T21:37:08Z

+numpy:
+    - '2'


That only will work when ##202 is merged

Agreed, this should land after #202. With the current pin_compatible('numpy', min_pin="x.x"), the 3.15 package built here requires numpy >=2.5,<3.0a0 at run time, which #202 fixes. Once #202 merges I will rebase: numpy: '2' will come from #202, and 3.10 will leave the python: list along with its matrix.

antonwolfy · 2026-10-26T21:45:27Z

@@ -1,3 +1,16 @@
+# The Python versions to build, one per line and without selectors: ci/matrix.py
+# reads them for .github/workflows/intel-ci.yml. NumPy 2, as in conda-forge's
+# global pinning; conda-package-cf.yml passes its own with --python and --numpy.


--numpy is dropping in #202

Removed from the comment in 6006c16, so it stays true after #202.

antonwolfy · 2026-10-26T21:47:12Z

+        "--artifacts", required=True, type=Path, help="the downloaded artifact"
+    )
+    parser.add_argument(
+        "--work", required=True, type=Path, help="an empty scratch directory"


hooks.py doesn't check that --work is empty, and may install conda-index into the user's base environment.

Fixed in b758764. hooks.py now refuses a --work that is not an empty directory, and it writes the local channel's repodata.json itself, as Jenkins does, so it installs nothing into base.

antonwolfy · 2026-10-26T21:48:47Z

+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.12"


Is there any issue to use the newer python?

No issue: the job only runs ci/matrix.py, which needs nothing beyond the standard library. Moved to 3.14 in 6006c16.

antonwolfy · 2026-10-26T21:53:59Z

+          activate-environment: ""
+          conda-remove-defaults: "true"
+
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0


Alternatively we can drop setup-python and run with shell: bash -el {0} with base activated.

bash is available on Windows runners too, and setup-miniconda supports this shell there. The step bodies become plain python ci/build.py … and python ci/hooks.py …. The $CONDA path code and hooks.py's re-run of itself would then only matter for local use.

Done in 6006c16: build and test run in bash -el {0} with base active, without setup-python. On Windows the new steps work: conda-build installs, the package builds and the imports pass. This run is still red on Windows because conda-forge's pytest 9.2.0, published today, dropped colorama again, so the recipe's pip check fails (conda-forge/pytest-feedstock#208). I will re-run once a fixed build is out.

Re-ran after conda-forge published the fixed pytest build: the run is green on Linux and Windows, test jobs included.

antonwolfy · 2026-10-26T22:00:21Z

@@ -0,0 +1,135 @@
+# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every


The same PR is open in three repos at once.
Have you considered moving common things (scripts, workflow) to some common repo like https://github.com/IntelPython/devops-tools ?
That would help to reduce the duplication.

Yes, we considered it. Duplicated code is a known consequence of the new CI's design, where each repository carries everything its CI needs, and the design's other benefits make up for it.

Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to the default branch no longer cancel each other. Same as IntelPython/mkl_random#204.

Adds what Intel Python CI (gh-ci) needs: a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds conda-recipe-cf with conda-build and the wheels with cibuildwheel, for every platform and Python version the recipe and pyproject.toml declare, and tests each package in a clean environment with ci/test.py. Intel's Jenkins tests them again, scans them, runs ci/benchmark.py on the newest Linux package, and publishes a release tag's tested packages as its GitHub Release. conda-recipe-cf/ is now the only recipe: conda-recipe/ and conda-package.yml, which built it against Intel's channel, are removed. conda-package-cf.yml reports the same check names. conda_build_config.yaml now lists the Python versions, and numpy: '2', as conda-build otherwise defaults to 1.26. The wheels' run-time requirement on mkl comes from [tool.gh-ci] wheel-requires in pyproject.toml, which ci/build.py adds to each wheel: in [project] it would fail the recipe's pip check, as conda-forge's mkl has no pip metadata.

jharlow-intel marked this pull request as ready for review October 23, 2026 20:42

jharlow-intel requested review from antonwolfy, ndgrigorian, vlad-perevezentsev and xaleryb as code owners October 23, 2026 20:42

antonwolfy added this to the 1.7.0 release milestone Oct 26, 2026

antonwolfy reviewed Oct 26, 2026

View reviewed changes

jharlow-intel self-assigned this Oct 28, 2026

jharlow-intel requested a review from antonwolfy October 28, 2026 18:31

jharlow-intel mentioned this pull request Oct 28, 2026

Build and test conda-recipe-cf for Intel Python CI IntelPython/mkl-service#265

Open

jharlow-intel force-pushed the ci/intel-python-ci branch 6 times, most recently from 8cda0c8 to 354eaf5 Compare October 30, 2026 16:37

jharlow-intel force-pushed the ci/intel-python-ci branch from 354eaf5 to 2655f3f Compare October 30, 2026 18:36

	cancel-in-progress: true
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

		@@ -0,0 +1,135 @@
		# Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every

Conversation

jharlow-intel commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Verification

Not verified

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jharlow-intel commented Oct 23, 2026 •

edited

Loading