AWS Introduces Push‑Based KV Transfer via NIXL to Accelerate vLLM Inference
Current overview (not historical)
AWS has unveiled a push‑based key‑value (KV) transfer feature built on NIXL, aimed at reducing tail‑latency and improving throughput for distributed inference in vLLM. In benchmark tests on Amazon EC2 P5en instances connected through Elastic Fabric Adapter (EFA), the new approach delivered up to a three‑fold reduction in time‑to‑first‑token (TTFT) and a 30 % improvement in tokens‑per‑second‑throughput (TPOT) for the Qwen3‑32B model, outperforming the traditional pull‑based method. The NIXL‑powered push model shifts the responsibility of KV transfer from the inference worker to a dedicated transfer service, cutting critical path delays and enabling more efficient use of GPU memory. While the initial results focus on a single large‑language‑model workload, AWS indicates that the technique is broadly applicable to other vLLM deployments. Overall, the feature represents a significant step toward faster, more scalable inference in cloud‑native AI workloads.
Written with AI from the reporting · updated 10/19/2026
EventGraph Explorer
One time cutoff for reports, relations and evidence.
New at this step ·AWS Introduces Push‑Based KV Transfer for vLLM to Cut TTFT and TPOT
- Reports in replay
- 1
- First-party
- 1
- Sources
- 1
Reports first seen at this step · 1
- AWS Machine Learning Blog · First-partyAWS Introduces Push‑Based KV Transfer for vLLM to Cut TTFT and TPOTOriginal source ↗
Discovery-time replay; the linked graph and evidence share this cutoff. The full overview and activity remain current.
Event developments
1 developments10/20/2026, 12:10:40 AM
AWS Introduces Push‑Based KV Transfer for vLLM to Cut TTFT and TPOTAWS Machine Learning Blog
Filtered by first discovery; only representative reports already discovered by this cutoff are shown. Story grouping may have changed later, so this is not a complete historical snapshot.
Relations at this time
Loading time-scoped relations…
Bounded reconstruction, not an exact snapshot: entity names, relationship text and story grouping may include later revisions. Overview and full source archive below remain current.
Story route
A station-by-station route from the first public evidence onward; every station carries its own evidence and uncertainty.
- 1Oct 20, 00:10single sourcereport-time only
AWS发布基于NIXL的推模式KV转移功能，提升vLLM分布式推理TTFT和TPOT
Read original source ↗ AWS Machine Learning BlogReplay this moment in the explorer
Uncertainty marks flag where to check the sources; they are not proven causation.
Full source archive (current)
Current public reports; outside the historical replay.
First-party sources · 1 reports
- AWS Machine Learning Blog · Oct 20, 00:05AWS Introduces Push‑Based KV Transfer for vLLM to Cut TTFT and TPOT
AWS has added a push‑based NIXL connector to the open‑source vLLM inference engine, moving KV cache transfer off the critical path between prefill and decode phases. In benchmarks on EC2 P5en instances linked with EFA, the push mode achieved up to three‑fold faster time‑to‑first‑token and 30% lower time‑per‑output‑token for the Qwen3‑32B model compared to the default pull mode. The improvement is driven by eliminating sequential coordination steps that previously extended the TTFT path.
Read original source ↗
Independent coverage · 0 reports
No verifiable reports in this group yet.
Different reports do not necessarily disagree. Check original sources and publication dates.
Report timeline
Explore every side through dated source reports.
- AWS Machine Learning BlogSelectedAWS Introduces Push‑Based KV Transfer for vLLM to Cut TTFT and TPOT
AWS has added a push‑based NIXL connector to the open‑source vLLM inference engine, moving KV cache transfer off the critical path between prefill and decode phases. In benchmarks on EC2 P5en instances linked with EFA, the push mode achieved up to three‑fold faster time‑to‑first‑token and 30% lower time‑per‑output‑token for the Qwen3‑32B model compared to the default pull mode. The improvement is driven by eliminating sequential coordination steps that previously extended the TTFT path.