Skip to content

Conversation

@evberrypi
Copy link
Contributor

@evberrypi evberrypi commented Nov 7, 2025

Purpose

Added support for multi-NIC configurations by parsing VLLM_HOST_IP from additional arguments. Built Ray IP environment variables to ensure correct network interface binding.

Test Plan

Run on device with multiple interfaces (such as DGX Spark) . The desired behavior is that the IP address of the NICs can be specified (previously they default to the IP of the host)
bash run_cluster.sh $VLLM_IMAGE 192.168.100.10 --head ~/.cache/huggingface
-e VLLM_HOST_IP=192.168.100.10
-e UCX_NET_DEVICES=$MN_IF_NAME
-e NCCL_SOCKET_IFNAME=$MN_IF_NAME
-e OMPI_MCA_btl_tcp_if_include=$MN_IF_NAME
-e GLOO_SOCKET_IFNAME=$MN_IF_NAME
-e TP_SOCKET_IFNAME=$MN_IF_NAME
-e RAY_memory_monitor_refresh_ms=0
-e MASTER_ADDR=192.168.100.10

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • [] The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Added support for multi-NIC configurations by parsing VLLM_HOST_IP from additional arguments. Built Ray IP environment variables to ensure correct network interface binding.

Signed-off-by: Ev Lacey <elacey@nvidia.com>
@github-actions
Copy link

github-actions bot commented Nov 7, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@mergify
Copy link

mergify bot commented Nov 7, 2025

Documentation preview: https://vllm--28328.org.readthedocs.build/en/28328/

@mergify mergify bot added the documentation Improvements or additions to documentation label Nov 7, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances run_cluster.sh to support multi-NIC configurations by allowing VLLM_HOST_IP to be specified. The core change involves parsing this IP from the command-line arguments. My review identifies a potential bug in the parsing logic which could lead to incorrect behavior if arguments are passed in an unexpected order. I've provided a more robust implementation to address this.

Comment on lines +88 to +97
VLLM_HOST_IP=""
for arg in "${ADDITIONAL_ARGS[@]}"; do
if [[ $arg == "-e" ]]; then
continue
fi
if [[ $arg == VLLM_HOST_IP=* ]]; then
VLLM_HOST_IP="${arg#VLLM_HOST_IP=}"
break
fi
done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current logic for parsing VLLM_HOST_IP is not robust. It will find and use a VLLM_HOST_IP=... argument even if it is not preceded by a -e flag, which is required for passing environment variables to docker run. This could lead to misinterpreting arguments intended for the command inside the container. The parsing should be stricter to only identify VLLM_HOST_IP when it's explicitly passed as an environment variable.

Suggested change
VLLM_HOST_IP=""
for arg in "${ADDITIONAL_ARGS[@]}"; do
if [[ $arg == "-e" ]]; then
continue
fi
if [[ $arg == VLLM_HOST_IP=* ]]; then
VLLM_HOST_IP="${arg#VLLM_HOST_IP=}"
break
fi
done
VLLM_HOST_IP=""
prev_arg=""
for arg in "${ADDITIONAL_ARGS[@]}"; do
if [[ "${prev_arg}" == "-e" ]] && [[ "${arg}" == VLLM_HOST_IP=* ]]; then
VLLM_HOST_IP="${arg#VLLM_HOST_IP=}"
break
fi
prev_arg="${arg}"
done

@mgoin mgoin added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 8, 2025
@mgoin mgoin enabled auto-merge (squash) November 8, 2025 20:57
@mgoin mgoin merged commit 77d702a into vllm-project:main Nov 8, 2025
24 checks passed
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Nov 13, 2025
Signed-off-by: Ev Lacey <elacey@nvidia.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
devpatelio pushed a commit to SumanthRH/vllm that referenced this pull request Nov 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants