Sync eng/common directory with azure-sdk-tools for PR 16685 - #3239
Open
azure-sdk-automation[bot] wants to merge 2 commits into
Open
Sync eng/common directory with azure-sdk-tools for PR 16685#3239azure-sdk-automation[bot] wants to merge 2 commits into
azure-sdk-automation[bot] wants to merge 2 commits into
Conversation
|
Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot could not run the full agentic suite for this review because it was automatically requested on a bot-authored pull request. Request a review from Copilot under Reviewers to retry with the full agentic suite. Improved support for bot-authored pull requests is coming soon.
Adds shared “eval-scripts” infrastructure under eng/common to support Vally eval CI, including a pinned Vally CLI install and a shard job template for running/publishing per-shard results.
Changes:
- Introduces a new npm workspace/package for eval CI glue scripts with a pinned
@microsoft/vally-cli. - Adds documentation describing the no-build TypeScript approach and how to update the pinned CLI version.
- Adds an Azure Pipelines job template to run eval shards, install the pinned CLI, and publish JUnit/transcript artifacts.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| eng/common/scripts/eval/package.json | Defines the eval-scripts package, Node/npm engine constraints, and pins @microsoft/vally-cli. |
| eng/common/scripts/eval/README.md | Documents purpose, TypeScript runtime constraints, vendored files, and update workflow for the pinned CLI. |
| eng/common/pipelines/templates/jobs/eval-shard.yml | Adds shard job template: sets up Node/npm, installs pinned deps via npm ci, primes fixtures, runs eval, and publishes artifacts. |
Files not reviewed (1)
- eng/common/scripts/eval/package-lock.json: Generated file
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+81
to
+83
| - script: npm ci --userconfig $(Build.SourcesDirectory)/.npmrc | ||
| displayName: 'Install Vally CLI + copilot-sdk (pinned)' | ||
| workingDirectory: eng/common/scripts/eval |
Comment on lines
+41
to
+42
| # A red shard shouldn't abort its siblings; the rollup gates in Summary. | ||
| continueOnError: true |
Comment on lines
+143
to
+144
| displayName: 'Collect $(shardName) transcripts (on failure)' | ||
| condition: failed() |
Comment on lines
+128
to
+132
| - script: | | ||
| mkdir -p "$(Build.ArtifactStagingDirectory)/junit" "$(Build.ArtifactStagingDirectory)/debug" | ||
| find "$(Build.SourcesDirectory)/artifacts/vally-results/$(shardName)" -name '*.junit.xml' -exec cp {} "$(Build.ArtifactStagingDirectory)/junit/" \; | ||
| displayName: 'Collect $(shardName) JUnit' | ||
| condition: always() |
Comment on lines
+15
to
+18
| The `*.ts` sources run directly through Node's native type stripping (erasable syntax only — | ||
| no `enum`/`namespace`/parameter properties, no emit). CI pins Node `22.x`, which strips types | ||
| unflagged on `>=22.18`; the pipeline `node` invocations and the `npm test` script pass | ||
| `--experimental-strip-types` so the same sources also run on older local Node (`>=22.6`), which |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync eng/common directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16685 See eng/common workflow