Skip to content

[Performance][gpt-oss] Revert gpt-oss max cudagraph size to 1024 #28345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 8, 2025
+4 −6
Merged

[Performance][gpt-oss] Revert gpt-oss max cudagraph size to 1024 #28345

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
26f1b1e
Revert gpt-oss max cudagraph size to 1024
mmangkad Nov 8, 2025
4489bae
Remove comment about max supported size
mmangkad Nov 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions vllm/model_executor/models/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -258,9 +258,9 @@ def verify_and_update_config(vllm_config: "VllmConfig") -> None:
if structured_outputs_config.reasoning_parser == "":
structured_outputs_config.reasoning_parser = "openai_gptoss"

# Increase the max capture size from 512 to 992 for performance.
# Increase the max capture size from 512 to 1024 for performance.
# NOTE(woosuk): This will increase the number of CUDA graphs
# from 67 to 81.
# from 67 to 83.
compilation_config = vllm_config.compilation_config
# Only override when the user has not set either of
# cudagraph_capture_sizes or max_cudagraph_capture_size.
Expand All @@ -269,10 +269,10 @@ def verify_and_update_config(vllm_config: "VllmConfig") -> None:
and compilation_config.max_cudagraph_capture_size is None
):
# FIXME(woosuk): When using full cuda graph with FA3, the max
# supported size is 992.
compilation_config.max_cudagraph_capture_size = 992
# supported size is 1024.
compilation_config.max_cudagraph_capture_size = 1024
logger.info(
"Overriding max cuda graph capture size to %d for performance.", 992
"Overriding max cuda graph capture size to %d for performance.", 1024
)


Expand Down