Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you |
|
@claude update PR description and fix PR place holder in perf-changelog.yaml file |
|
Claude finished @Ankur-singh's task in 1m 7s —— View job Task: Update PR description and fix perf-changelog.yaml
Done
|
Co-authored-by: Ankur Singh <Ankur-singh@users.noreply.github.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| --max-model-len $MAX_MODEL_LEN \ | ||
| --block-size=32 \ | ||
| --kv-cache-dtype fp8 \ | ||
| --stream-interval 20 --no-enable-prefix-caching \ |
There was a problem hiding this comment.
🔴 The script passes --stream-interval 20 to vllm serve, which is a SGLang-specific argument not recognized by vLLM; this will cause the server to exit immediately with an 'unrecognized arguments: --stream-interval' error, preventing the benchmark from running at all. Remove --stream-interval 20 from the vllm serve invocation — the companion FP8 script (minimaxm2.5_fp8_b200.sh) omits this flag and serves as the correct reference.
Extended reasoning...
Bug: --stream-interval is a SGLang-only argument passed to vllm serve
What the bug is and how it manifests
Line 50 of benchmarks/single_node/minimaxm2.5_fp4_b200.sh passes --stream-interval 20 as part of the vllm serve command. This flag is a SGLang server parameter (it controls how often SGLang flushes streamed tokens) and is not part of the vLLM CLI argument set. When vLLM receives an unrecognized argument it calls argparse's standard error path, prints 'unrecognized arguments: --stream-interval', and exits with a non-zero status before the server ever starts.
The specific code path
The relevant section of the script is:
vllm serve $MODEL --port $PORT
--tensor-parallel-size=$TP
$EP
--gpu-memory-utilization 0.95
--max-model-len $MAX_MODEL_LEN
--block-size=32
--kv-cache-dtype fp8
--stream-interval 20 --no-enable-prefix-caching <-- line 50: invalid for vLLM
--trust-remote-code > $SERVER_LOG 2>&1 &
After vllm serve exits, wait_for_server_ready will poll until the timeout expires (or detect the dead PID), and the benchmark job fails without producing any results.
Why existing code does not prevent it
No argument validation is performed in benchmark_lib.sh or any wrapper script — arguments are passed verbatim to the underlying framework binary. The flag was almost certainly copy-pasted from a SGLang benchmark script. Every other occurrence of --stream-interval in the repo (dsr1_fp4_b200.sh, dsr1_fp8_b200.sh, glm5_nvfp4_b200.sh, qwen3.5_bf16_b200.sh, etc.) is in a script that launches python3 -m sglang.launch_server, not vllm serve. A targeted search for 'vllm serve.*stream-interval' returns zero results across the entire benchmarks/ directory.
Step-by-step proof
- A CI job picks up minimaxm2.5-fp4-b200-vllm from nvidia-master.yaml.
- The harness executes minimaxm2.5_fp4_b200.sh inside the vLLM nightly container.
- The shell reaches the vllm serve block and starts the server process.
- vLLM parses its CLI arguments; --stream-interval is not in vllm serve's argparse namespace.
- vLLM prints: 'error: unrecognized arguments: --stream-interval' and exits non-zero.
- wait_for_server_ready detects that SERVER_PID died (or times out polling the health endpoint) and terminates the script with an error — no benchmark results are produced.
How to fix it
Simply remove --stream-interval 20 from the vllm serve invocation on line 50. The direct reference implementation benchmarks/single_node/minimaxm2.5_fp8_b200.sh (the FP8 B200 companion) uses vllm serve without this flag and is the correct pattern to follow.
perf-changelog.yaml
Outdated
| description: | ||
| - "MiniMax-M2.5 NVFP4 benchmark config for B200" | ||
| - "Uses nvidia/MiniMax-M2.5-NVFP4 model checkpoint" | ||
| - "Image: vllm/vllm-openai:nightly-5b8c30d62b754b575e043ce2fc0dcbf8a64f6306" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/996 |
There was a problem hiding this comment.
🟡 The new perf-changelog.yaml entry for minimaxm2.5-fp4-b200-vllm has pr-link: .../pull/xxx — a lowercase placeholder that was never updated to the actual PR number (996). After merge, the changelog link will point to a non-existent URL.
Extended reasoning...
What the bug is: The changelog entry added by this PR ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/xxx. The lowercase xxx is clearly a placeholder that was never replaced with the actual PR number before submission.
The specific code path: In perf-changelog.yaml at lines 1241-1245, the newly added entry for minimaxm2.5-fp4-b200-vllm reads:
- config-keys:
- minimaxm2.5-fp4-b200-vllm
description:
- "MiniMax-M2.5 NVFP4 benchmark config for B200"
- "Uses nvidia/MiniMax-M2.5-NVFP4 model checkpoint"
- "Image: vllm/vllm-openai:nightly-5b8c30d62b754b575e043ce2fc0dcbf8a64f6306"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/xxxWhy existing code doesn't prevent it: The changelog file is manually maintained YAML with no automated validation of pr-link values. There's no CI check that verifies placeholder values aren't committed.
Impact: Once merged, anyone referencing the changelog entry for minimaxm2.5-fp4-b200-vllm will follow a broken link. The correct URL should be https://github.com/SemiAnalysisAI/InferenceX/pull/996. It's worth noting that 7 other entries in the same file also use uppercase XXX placeholders — making this a broader documentation hygiene issue — but the present PR introduces a new instance with lowercase xxx that can be fixed now since the PR number is known.
How to fix: Replace pull/xxx with pull/996 in the new entry.
Step-by-step proof:
- This PR is numbered 996 (visible in PR metadata:
<pr number="996">). - The diff adds a new entry to perf-changelog.yaml with
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/xxx. - Navigating to
https://github.com/SemiAnalysisAI/InferenceX/pull/xxxafter merge would return a 404 (or redirect to a search page), sincexxxis not a valid PR number. - The fix is trivial: change
xxx→996in that one line.
| start_gpu_monitor | ||
|
|
||
| set -x | ||
| vllm serve $MODEL --port $PORT \ |
There was a problem hiding this comment.
add nvfp4 minimax to vllm recipes plz
| - { tp: 4, conc-start: 4, conc-end: 64 } | ||
|
|
||
| minimaxm2.5-fp4-b200-vllm: | ||
| image: vllm/vllm-openai:nightly-5b8c30d62b754b575e043ce2fc0dcbf8a64f6306 |
There was a problem hiding this comment.
0.19 comes out tmw btw
Ankur-singh
force-pushed
the
minimaxm2.5-nvfp4-b200
branch
from
31510da to
08bcde8
Compare
3 participants
Summary
Add MiniMax-M2.5 NVFP4 benchmark configuration and script for B200 GPUs using vLLM.
Changes
.github/configs/nvidia-master.yaml): Addedminimaxm2.5-fp4-b200-vllmconfig usingnvidia/MiniMax-M2.5-NVFP4model with vLLM nightly image (vllm/vllm-openai:nightly-5b8c30d62b754b575e043ce2fc0dcbf8a64f6306). Supports TP=2 and TP=4 with concurrency range 4–256 at 1k/1k and 8k/1k sequence lengths.benchmarks/single_node/minimaxm2.5_fp4_b200.sh): New vLLM serve script with FP8 KV cache, expert parallelism support, andVLLM_FLASHINFER_ALLREDUCE_BACKEND=mnnvl.perf-changelog.yaml): Added entry for the new config.Test Plan
minimaxm2.5-fp4-b200-vllmon B200 to validate config