Skip to content

Commit f7050a9

Browse files
adobrzynCopilot
andauthored
[Bucketing] Prompt with 0 min and max context blocks (#534)
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai> Signed-off-by: Agata Dobrzyniewicz <160237065+adobrzyn@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent f42b34b commit f7050a9

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm_gaudi/extension/bucketing/linear.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,8 @@ def warmup_range(config: Tuple[int, int, int]):
132132
"batch size. If you want to skip warmup, "
133133
"set VLLM_SKIP_WARMUP=true")
134134
if add_zero_bucket:
135+
if bmin == 0 and bmax == 0:
136+
return [0]
135137
bmin = bstep
136138
base = itertools.repeat(2)
137139
ramp_up_acc = itertools.accumulate(base, func=operator.mul, initial=bmin)

0 commit comments

Comments
 (0)