vLLM Boosts Multi‑Turn Agent Workloads with Bi‑Directional KV Transfer
Current overview (not historical)
vLLM has added a bi‑directional key‑value (KV) transfer mechanism to its P‑D distributed deployment. The new approach keeps KV caches across decoding rounds and sends them back to the prefill node for reuse, eliminating redundant computation. Early reports claim that on Qwen3‑32B, the time to first token (TTFT) can be reduced by up to 1.6× under the longest prompt, while on GLM‑5‑FP8 the reduction reaches 3.3×. These gains demonstrate the potential of KV cache sharing to accelerate multi‑turn agentic workloads, though the exact performance may vary with model size and deployment configuration. The technique relies on maintaining KV states at the decode node and transmitting them back to the prefill node, a departure from the traditional one‑way flow. While the initial findings are promising, further benchmarking across diverse models and workloads is needed to confirm the scalability and robustness of the approach.
Written with AI from the reporting · updated 10/19/2026
EventGraph Explorer
One time cutoff for reports, relations and evidence.
New at this step ·vLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic Workloads
- Reports in replay
- 1
- First-party
- 1
- Sources
- 1
Reports first seen at this step · 1
- AWS Machine Learning Blog · First-partyvLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic WorkloadsOriginal source ↗
Discovery-time replay; the linked graph and evidence share this cutoff. The full overview and activity remain current.
Event developments
1 developments10/20/2026, 12:10:40 AM
vLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic WorkloadsAWS Machine Learning Blog
Filtered by first discovery; only representative reports already discovered by this cutoff are shown. Story grouping may have changed later, so this is not a complete historical snapshot.
Relations at this time
Loading time-scoped relations…
Bounded reconstruction, not an exact snapshot: entity names, relationship text and story grouping may include later revisions. Overview and full source archive below remain current.
Story route
A station-by-station route from the first public evidence onward; every station carries its own evidence and uncertainty.
- 1Oct 20, 00:10single sourcereport-time only
vLLM 引入双向 KV 传输以提升多轮代理工作负载 TTFT
Read original source ↗ AWS Machine Learning BlogReplay this moment in the explorer
Uncertainty marks flag where to check the sources; they are not proven causation.
Full source archive (current)
Current public reports; outside the historical replay.
First-party sources · 1 reports
- AWS Machine Learning Blog · Oct 20, 00:03vLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic Workloads
vLLM now supports bi‑directional key‑value (KV) cache transfer between prefill and decode nodes in its Prefill‑Decode (P‑D) architecture. By allowing the decode node to send back KV blocks for previously generated tokens, the system eliminates redundant prefill computations on subsequent turns, reducing time‑to‑first‑token (TTFT). Benchmarks on an Amazon EC2 P5en GPU cluster with EFA show TTFT reductions of up to 1.6× for Qwen3‑32B and 3.3× for zai‑org/GLM‑5‑FP8. The feature is accelerator‑agnostic and leverages GPUDirect RDMA or Neuron Direct RDMA for low‑latency KV movement.
Read original source ↗
Independent coverage · 0 reports
No verifiable reports in this group yet.
Different reports do not necessarily disagree. Check original sources and publication dates.
Report timeline
Explore every side through dated source reports.
- AWS Machine Learning BlogSelectedvLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic Workloads
vLLM now supports bi‑directional key‑value (KV) cache transfer between prefill and decode nodes in its Prefill‑Decode (P‑D) architecture. By allowing the decode node to send back KV blocks for previously generated tokens, the system eliminates redundant prefill computations on subsequent turns, reducing time‑to‑first‑token (TTFT). Benchmarks on an Amazon EC2 P5en GPU cluster with EFA show TTFT reductions of up to 1.6× for Qwen3‑32B and 3.3× for zai‑org/GLM‑5‑FP8. The feature is accelerator‑agnostic and leverages GPUDirect RDMA or Neuron Direct RDMA for low‑latency KV movement.