Feat/over saturation stopping #438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

AlonKellner-RedHat wants to merge 36 commits into vllm-project:main from AlonKellner-RedHat:feat/over-saturation-stopping

Collaborator

AlonKellner-RedHat commented Oct 29, 2025 •

edited

Loading

Summary

This PR adds over-saturation stopping to the GuideLLM CLI.
It's based on the OSD (Over-Saturation Detection) algorithm we developed and evaluated at Jounce.
Use --stop-over-saturated or --stop-osd to enable.

Details

This PR adds:

Over-saturation stopping (--stop-over-saturated)
Comprehensive OSD unit tests

Test Plan

Currently, only unit tests
When test(e2e): basic E2Es #440 lands, we'll enable its over-saturation stopping E2E test

"I certify that all code in this PR is my own, except as noted below."

Use of AI

Includes AI-assisted code completion
Includes code generated by an AI application
Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

AlonKellner-RedHat mentioned this pull request

test(e2e): basic E2Es #440

Merged

9 tasks

AlonKellner-RedHat requested review from markurtz and sjmonson

November 4, 2025 16:46

AlonKellner-RedHat force-pushed the feat/over-saturation-stopping branch 3 times, most recently from 65bf98c to 5b4dcca Compare

November 6, 2025 11:02

AlonKellner-RedHat and others added 24 commits

November 6, 2025 11:07


          feat: over-saturation stopping

694019a

Signed-off-by: Alon Kellner <akellner@redhat.com>


          test: over-saturation stopping

d18f1fa

Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix: remove unused diff patch

7d64e02

Signed-off-by: Alon Kellner <akellner@redhat.com>


          test: comprehensive over-saturation detection

76d6090

Signed-off-by: Alon Kellner <akellner@redhat.com>


          test: formatting

c422313

Signed-off-by: Alon Kellner <akellner@redhat.com>


          Attempt to parse sythetic config with model validate

9cb47ee

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Only set args if they are manually set on the commandline

b3e5e6e

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Convert single data to list

d28f4c6

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Add list conversion for more fields

defdea1

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Add -c alias for scenario

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Support dashed arguments for benchmark args

2b63eb3

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Fix commandline arguments not overriding scenario if set to default

ce9384b

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Change request_type precedence

bcad8a9

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix: constraints package

2c4c046

Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix test_output xfail ut case

2d3114e

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix some code quality check error

dc96b69

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          remove marker xfail

39371bf

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          update benchmark mocker requests

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix(pre-commit): autofixes

dd00f4f

Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix(pre-commit): mdformat

ad17ddc

Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix(test): import error

4bdb1ba

Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix(pdm): new aiologic version breaks culsans

f3d8d1d

Signed-off-by: Alon Kellner <akellner@redhat.com>


          update prod ui version

62fed58

Signed-off-by: dalthecow <dalcowboiz@gmail.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix schedule xfail ut case

2b1db8b

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>

tukwila and others added 9 commits

November 6, 2025 11:07


          update test_main xfail ut case

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          update staging version to v0.3.1

b373bc4

Signed-off-by: dalthecow <dalcowboiz@gmail.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix: removed aiologic pinning

a5422b0

Signed-off-by: Alon Kellner <akellner@redhat.com>


          Convert single data to list

6f847cf

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Add list conversion for more fields

427f151

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          fix test_output xfail ut case

1c530f1

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          update benchmark mocker requests

df37695

Signed-off-by: guangli.bao <guangli.bao@daocloud.io>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Support dashed arguments for benchmark args

17f469b

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>


          Change request_type precedence

f996254

Signed-off-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Alon Kellner <akellner@redhat.com>

AlonKellner-RedHat force-pushed the feat/over-saturation-stopping branch from 85cf65e to f996254 Compare

November 6, 2025 11:11


          Merge branch 'main' into feat/over-saturation-stopping

0c0b2ed

sjmonson added a commit that referenced this pull request


          test(e2e): basic E2Es (#440)

f59a60f

## Summary

E2E tests which check basic GuideLLM functionality, using vLLM
simulator.

## Details

- [x] Max requests test
- [x] Max duration test
- [ ] Over-saturation stopping test - skipped for now, will be enabled
when #438 lands

## Test Plan

- [x] Local testing
- [x] GitHub action

---

- [x] "I certify that all code in this PR is my own, except as noted
below."

## Use of AI

- [x] Includes AI-assisted code completion
- [ ] Includes code generated by an AI application
- [ ] Includes AI-generated tests (NOTE: AI written tests should have a
docstring that includes `## WRITTEN BY AI ##`)

AlonKellner-RedHat added 2 commits

November 6, 2025 18:32


          Merge branch 'main' into feat/over-saturation-stopping

607d266

Signed-off-by: Alon Kellner <akellner@redhat.com>


          test(e2e): enable over-saturation test

30f96f7

Signed-off-by: Alon Kellner <akellner@redhat.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet