A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing

(arxiv.org)

4 points | by 1a1a11a 8 hours ago ago