[translation-bench] Part 1/n: scaffold + generate price snapshots + catalogs - #2771
Merged
Conversation
Add @typeagent/benchmarks package layout with core/paths, minimal types, catalog loader, and scripts to regenerate catalog.generated.json and OpenAI-only model-prices.generated.json from models.dev.
Match package root by package name, remove BENCH_RESULTS_DIR override, and exclude gpt-5.4-pro from the OpenAI model price snapshot.
Resolve package root via two fixed parent candidates and the package name check instead of walking the filesystem or a self-require that pnpm does not link.
Use getPackageFilePath like cache/defaultAgentProvider, emit under dist/ via rootDir src, move asset copy/clean into scripts/, and drop unused gpt-5.6 / gpt-5.4-pro price entries.
Also remove tsconfig.tsbuildinfo; otherwise incremental build skips emit.
datduyng
changed the title
Local install had rewritten lockfile resolutions with ms-feed tarball URLs and sha1 digests, which fail supply-chain verification on CI. Restore main lockfile style and only add the benchmarks importer.
Repo root MIT license covers the package; no per-package copy needed.
Drop prefix/include/exclude matching; pin the exact model ids we price.
jebrans
reviewed
Jul 31, 2026
jebrans
approved these changes
Jul 31, 2026
datduyng
enabled auto-merge
July 31, 2026 17:14
Type the loose action-schema tree with local interfaces so CI no-explicit-any does not regress on the new package.
Copilot stopped work on behalf of
datduyng due to an error
July 31, 2026 19:36
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 31, 2026
robgruen
reviewed
Jul 31, 2026
| const packageRoot = path.join("..", ".."); | ||
|
|
||
| /** Resolve a path relative to the package root (works from compiled dist/). */ | ||
| export function getPackageFilePath(packageRootRelativePath: string): string { |
Collaborator
There was a problem hiding this comment.
We have this function in several places. We should consolidate them into the common or utils packages at some point.
robgruen
reviewed
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 of the translation-bench series: package scaffold and pinned generators only (no runner/matrix yet).