Skip to content

CAMEL-24318 - improve regen.sh - #25260

Merged
apupier merged 1 commit into
apache:mainfrom
apupier:useMulticoreToRegen
Jul 31, 2026
Merged

CAMEL-24318 - improve regen.sh#25260
apupier merged 1 commit into
apache:mainfrom
apupier:useMulticoreToRegen

Conversation

@apupier

@apupier apupier commented Jul 30, 2026

Copy link
Copy Markdown
Contributor
  • use multithreads for Maven to improve performance. locally with 22 cores, it divided the time by 2. On GitHub CI, it won't be that much as there is only 4 Cores currently
  • remove x option from git clean to avoid removing the files which are part of .gitignore, for instance all IDE related files
  • renamed the build log to avoid overriding the first one and to be able to keep both of them

Description

Target

  • I checked that the commit is targeting the correct branch (Camel 4 uses the main branch)

Tracking

  • If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

  • I checked that each commit in the pull request has a meaningful subject line and body.
  • I have run mvn clean install -DskipTests locally from root folder and I have committed all auto-generated changes.

AI-assisted contributions

  • If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., Co-authored-by trailers) and the PR description identifies the AI tool used.

@apupier
apupier force-pushed the useMulticoreToRegen branch from 8d5797a to 187812a Compare July 30, 2026 14:40
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@apupier
apupier marked this pull request as ready for review July 30, 2026 15:29
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • etc

🔬 Scalpel shadow comparison — Scalpel: 1 tested, 0 compile-only — current: 0 all tested

Maveniverse Scalpel detected 1 affected modules (current approach: 0).

⚠️ Modules only in Scalpel (1)
  • camel-etc

Skip-tests mode would test 1 modules (1 direct + 0 downstream), skip tests for 0 (generated code, meta-modules)

Modules Scalpel would test (1)
  • camel-etc

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more


⚙️ View full build and test results

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good improvements — multithreaded Maven builds and separate log files for regen are welcome changes. A few observations:

Missing pr-build-main.yml update:

The PR updates main-build.yml to reference the new log filenames (build-regen.log, build-regen-catalog.log), but .github/workflows/pr-build-main.yml still references build.log at line 148. When pr-build-main.yml runs the full-build path (line 135: ./etc/scripts/regen.sh), regen.sh will write to the new filenames, but the artifact upload step will look for the non-existent build.log. The upload-artifact action will warn rather than fail, but log artifacts will be missing from PR builds, making CI failures harder to debug. Consider applying the same log filename update to pr-build-main.yml.

Minor observations (not blocking):

  • Removing -x from git clean means .gitignore'd files (including *.log per line 13 of .gitignore) survive between runs. Combined with the append operator (>>), log files from previous local runs accumulate rather than starting fresh. In CI (fresh checkout) this is harmless. Consider adding rm -f build-regen.log build-regen-catalog.log before the Maven invocations, or switching from >> to >.
  • The -T1C flag on the second mvnw invocation (-f catalog/camel-catalog) targets a single POM, so Maven parallelism has nothing to distribute there. Harmless but unnecessary.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

Claude Code on behalf of Guillaume Nodet

@apupier
apupier force-pushed the useMulticoreToRegen branch from 187812a to 53f2a96 Compare July 31, 2026 06:18
@apupier
apupier requested a review from gnodet July 31, 2026 06:18

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code on behalf of Guillaume Nodet

Thanks for addressing all three points from the previous review — pr-build-main.yml is now updated, >> was changed to > to avoid log accumulation, and -T1C was correctly not added to the single-POM catalog build. The main-build.yml change is correct as-is.

One issue with the updated artifact upload paths:


🟡 MEDIUM — build.log dropped from artifact upload in pr-build-main.yml

The artifact upload (line ~148) now only lists build-regen.log and build-regen-catalog.log, but the "Quick dependency build" step (line ~142) still writes to build.log via Maven's -l build.log flag. When skip_full_build is true, regen.sh does not run, so the regen log files are never created — meanwhile build.log IS produced but is no longer uploaded.

This means no build logs are archived for quick PR builds, making CI failures harder to debug.

Suggested fix — add build.log to the upload path list:

path: |
  build-regen.log
  build-regen-catalog.log
  build.log

@apupier
apupier force-pushed the useMulticoreToRegen branch from 53f2a96 to 5a9967b Compare July 31, 2026 07:08
* use multithreads for Maven to improve performance. locally with 22
cores, it divided the time by 2. On GitHub CI, it won't be that much as
there is only 4 Cores currently
* remove x option from git clean to avoid removing the files which are
part of .gitignore, for instance all IDE related files
* renamed the build log to avoid overriding the first one and to be able
to keep both of them

Signed-off-by: Aurélien Pupier <apupier@ibm.com>
@apupier
apupier force-pushed the useMulticoreToRegen branch from 5a9967b to e1d86c9 Compare July 31, 2026 07:08
@apupier
apupier requested a review from gnodet July 31, 2026 07:08

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code on behalf of Guillaume Nodet

All previous feedback has been addressed — build.log is now included in the pr-build-main.yml artifact upload paths alongside the regen log files, covering both the quick-dependency-build path and the full-regen path. The upload-artifact v7 action gracefully handles missing files, so whichever build path runs, the relevant logs are captured.

The three changes across both workflow files are correct:

  • pr-build-main.yml updated with all three log files
  • >> changed to > to avoid log accumulation
  • -T1C correctly omitted from the single-POM catalog build
  • main-build.yml correctly omits build.log since it only runs regen.sh

@apupier
apupier merged commit f434910 into apache:main Jul 31, 2026
5 checks passed
@apupier
apupier deleted the useMulticoreToRegen branch July 31, 2026 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants