LLM KV Cache Systems Engineer
Huawei Hong Kong Research Center(HKRC)
AI assistant
Job Description
About the role
We are looking for an LLM KV Cache Systems Engineer to design and build KV Cache subsystems for large-language-model inference. You will work on memory allocation, lifecycle management, reuse, compression, transfer, tiering, and KV-aware scheduling across single-node and distributed serving systems.
This is a systems role. Direct source-level experience with an LLM KV Cache subsystem is more important than familiarity with serving-framework APIs alone.
What you will do
- Design block/page allocators, KV Cache lifecycle management, fragmentation control, eviction, prefix caching, cross-request reuse, and preemption/recovery mechanisms.
- Optimize KV layouts, GQA/MQA handling, quantization, compression, or sparsity while balancing accuracy, capacity, memory bandwidth, and compute overhead.
- Build KV transfer and disaggregated-serving mechanisms, including prefill/decode