Skip to content

feat: support Global Performance Tuning - #4476

Open
zewenli98 wants to merge 1 commit into
mainfrom
evanli/feat_gpt
Open

feat: support Global Performance Tuning#4476
zewenli98 wants to merge 1 commit into
mainfrom
evanli/feat_gpt

Conversation

@zewenli98

Copy link
Copy Markdown
Collaborator

Description

TensorRT's Global Performance Tuner searches internal builder knobs for faster engines. Torch-TensorRT exposes the same capability in-process for Dynamo TRT subgraphs. Please see the description for details.

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@zewenli98
zewenli98 requested a review from narendasan August 11, 2026 05:59
@zewenli98 zewenli98 self-assigned this Aug 11, 2026
@meta-cla meta-cla Bot added the cla signed label Aug 11, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation component: tests Issues re: Tests component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Aug 11, 2026
del trt_mod
torch.cuda.empty_cache()
_LOGGER.info(f"&&&& TASK_END [iter={iter_idx}] BuildRoute = '{route}'")
except Exception as exc:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we cannot guarantee the TensorRT will not crash when setting a build route, for example, some driver level exception might be raised and it will not be caught by the try-catch block, and probably will crash the entire Python program. So I suggest we spawn a new process like the implementation in trtexec, so that even if the new child process crashes, the parent process remains unaffected and can continue to the next iteration.

@zewenli98 zewenli98 added this to the v2.15.0 milestone Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: tests Issues re: Tests documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants