[WIP][experimental] add agentic trace replay benchmark infrastructure#993

Draft

cquil11 wants to merge 33 commits intomainfrom

experimental/agentic-benchmark

Commits on Apr 1, 2026

add agentic trace replay benchmark infrastructure

cquil11
and
claude
committed
remove deprecated GpuTransferCollector from metrics collector

cquil11
and
claude
committed
modularize metrics collector with backend auto-detection

cquil11
and
claude
committed
remove unused Protocol import
cquil11
committed
add LMCache agentic trace benchmark for H100

cquil11
and
claude
committed
add H100 LMCache trace sweep config

cquil11
and
claude
committed
fix LMCache benchmark: use fixed-schedule replay, remove ignore_eos

cquil11
and
claude
committed
remove --fixed-schedule: use concurrency mode per Samuel's recommendation
cquil11
committed

Commits on Apr 2, 2026

update yaml
cquil11
committed
fix H100 runner: add SCRIPT_SUFFIX support

cquil11
and
claude
committed
fix: mkdir RESULT_DIR before trace conversion
cquil11
committed
add H200 LMCache trace benchmark and config

cquil11
and
claude
committed
update yaml
cquil11
committed
fix H200-nb runner: add SCRIPT_SUFFIX support
cquil11
committed
fix all H200 runners: add SCRIPT_SUFFIX support
cquil11
committed
fix all runners: add SCRIPT_SUFFIX support

cquil11
and
claude
committed
reduce multiturn artifact size: upload only files needed for post-processing

cquil11
and
claude
committed
add exclusive
cquil11
committed
add exclusive
cquil11
committed
add exclusive
cquil11
committed
use aiperf summary CSV instead of per-record JSONL for post-processing

cquil11
and
claude
committed
debug
cquil11
committed
fix LMCache traces: convert system role to developer for vLLM v0.18+

cquil11
and
claude
committed
revert system->developer role conversion in LMCache traces

cquil11
and
claude
committed
fix MetricsCollector missing gpu_transfer_collector attribute

cquil11
and
claude
committed
fix LMCache traces: strip null fields to pass vLLM Pydantic validation

cquil11
and
claude
committed
use hf download for LMCache traces instead of datasets.load_dataset

cquil11
and
claude
committed
add B200 FP4 multiturn benchmark script using aiperf

cquil11
and
claude
committed
add entry for b200 ds
cquil11
committed
add expert parallel support to B200 FP4 aiperf script

cquil11
and
claude
committed
skip LMCache trace entries with empty messages

cquil11
and
claude
committed
fix: prioritize aiperf summary CSV over malformed client CSV

cquil11
and
claude
committed
fix aiperf CSV parser: handle multi-section format with different column counts

cquil11
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP][experimental] add agentic trace replay benchmark infrastructure#993

[WIP][experimental] add agentic trace replay benchmark infrastructure#993
cquil11 wants to merge 33 commits intomainfrom
experimental/agentic-benchmark

Commits on Apr 1, 2026

add agentic trace replay benchmark infrastructure

remove deprecated GpuTransferCollector from metrics collector

modularize metrics collector with backend auto-detection

remove unused Protocol import

add LMCache agentic trace benchmark for H100

add H100 LMCache trace sweep config

fix LMCache benchmark: use fixed-schedule replay, remove ignore_eos

remove --fixed-schedule: use concurrency mode per Samuel's recommendation

Commits on Apr 2, 2026

update yaml

fix H100 runner: add SCRIPT_SUFFIX support

fix: mkdir RESULT_DIR before trace conversion

add H200 LMCache trace benchmark and config

update yaml

fix H200-nb runner: add SCRIPT_SUFFIX support

fix all H200 runners: add SCRIPT_SUFFIX support

fix all runners: add SCRIPT_SUFFIX support

reduce multiturn artifact size: upload only files needed for post-processing

add exclusive

add exclusive

add exclusive

use aiperf summary CSV instead of per-record JSONL for post-processing

debug

fix LMCache traces: convert system role to developer for vLLM v0.18+

revert system->developer role conversion in LMCache traces

fix MetricsCollector missing gpu_transfer_collector attribute

fix LMCache traces: strip null fields to pass vLLM Pydantic validation

use hf download for LMCache traces instead of datasets.load_dataset

add B200 FP4 multiturn benchmark script using aiperf

add entry for b200 ds

add expert parallel support to B200 FP4 aiperf script

skip LMCache trace entries with empty messages

fix: prioritize aiperf summary CSV over malformed client CSV

fix aiperf CSV parser: handle multi-section format with different column counts