Article · MarkTechPost ·
Google Releases TPU Sync: Open-Source KV Cache Transfer for TPUs
Google released TPU Sync, an open-source C++ library that moves KV caches and weights across TPU HBM, host DRAM, storage and network. Google reports over 90% lower KV transfer latency on 235B to 480B MoE models.
In this story
- Repogoogle/tpu-sync
- Repogoogle-pytorch/torch_tpu
- Repovllm-project/vllm-torchtpu
- Reposgl-project/sglang-torchtpu
- Repollm-d/llm-d
- Repoai-dynamo/nixl
- Repolmcache/lmcache
- Repokvcache-ai/mooncake
A tool or model opens in the Heidelberg AI Curriculum: what it does, what it costs, how to start. A repo opens on GitHub.
