◈ ART & CULTURE · BENGALURU
The KV Cache Bottleneck: Architectural Alternatives
- When
- Wednesday, 21 October 2026, 6:30 pm
- Where
- Bengaluru, IndiaBengaluru, Karnataka
- Price
- Free
- Organiser
- Lossfunk Event Calendar
In autoregressive decoding, each new token recomputes the keys and values for all preceding tokens. This cache grows linearly with context length, and in long-context serving it becomes the dominant memory and bandwidth cost - quite often more limiting than raw compute itself. In this session, Abhay