[Model][Qwen3VL] Simplify get_mrope_input_positions using numpy

[lgeiger]

Purpose

This PR simplifies the Qwen3VL get_mrope_input_positions computation by using np.indices to make the code more readable.

In a profile I also noticed that the torch CPUs ops, especially torch.tensor(list[int]) are slower than their numpy equivalents so this PR also changes the computation to numpy.

Before:

After:

Test Plan

VLLM_WORKER_MULTIPROC_METHOD=spawn lm_eval --model vllm-vlm --model_args "pretrained=Qwen/Qwen3-VL-30B-A3B-Instruct-FP8,max_model_len=10000" --tasks chartqa --batch_size auto --apply_chat_template

Test Result

Before:

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
chartqa	0	none	0	anywhere_accuracy	↑	0.8680	±	0.0068
		none	0	exact_match	↑	0.6340	±	0.0096
		none	0	relaxed_accuracy	↑	0.8572	±	0.0070

After:

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
chartqa	0	none	0	anywhere_accuracy	↑	0.8672	±	0.0068
		none	0	exact_match	↑	0.6324	±	0.0096
		none	0	relaxed_accuracy	↑	0.8576	±	0.0070

[gemini-code-assist]

Code Review

This pull request refactors the get_mrope_input_positions function to use NumPy instead of PyTorch for improved performance and readability. The changes are logical and well-implemented, replacing complex PyTorch operations with more concise NumPy equivalents like np.indices.

I've identified one high-severity issue related to an edge case where empty input_tokens would cause a crash. I've provided a code suggestion to handle this case gracefully. Other than that, the changes look good.

[DarkLight1337]

Actually I'm thinking of directly using the positions from mm_features instead of having to calculate the mask again, WDYT?

[lgeiger]

Actually I'm thinking of directly using the positions from mm_features instead of having to calculate the mask again, WDYT?

I'm not entirely sure how you plan to do this, but not having to compute the mask again does sound sensible.

[DarkLight1337]

We can pass req_state.mm_features which includes mm_position (PlaceholderRange) into get_mrope_input_positions.

[DarkLight1337]

I will open another PR to update the argument list, then we can migrate the models to actually make use of mm_position one by one.

[DarkLight1337]

Opened #28399

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @lgeiger.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[DarkLight1337]

Feel free to update your PR now

[DarkLight1337]

Thanks, LGTM

[lgeiger]

@DarkLight1337 Just to make sure there's no confusion: I only rebased this PR so far but haven't made use of mm_position yet. The PR is still valid, though.

I'm not sure when I'll have time to migrate the logic to use mm_position but I'll try to have a look early next week. If you want to make this change soon feel free to make a PR otherwise I'll have a look at it on the weekend.

[DarkLight1337]

Feel free to work on this yourself!