vLLM Adds Bi‑Directional KV Transfer to Cut TTFT for Multi‑Turn Agentic Workloads

vLLM now supports bi‑directional key‑value (KV) cache transfer between prefill and decode nodes in its Prefill‑Decode (P‑D) architecture. By allowing the decode node to send back KV blocks for previously generated tokens, the system eliminates redundant prefill computations on subsequent turns, reducing time‑to‑first‑token (TTFT). Benchmarks on an Amazon EC2 P5en GPU cluster with EFA show TTFT reductions of up to 1.6× for Qwen3‑32B and 3.3× for zai‑org/GLM‑5‑FP8. The feature is accelerator‑agnostic and leverages GPUDirect RDMA or Neuron Direct RDMA for low‑latency KV movement.