LLM KV Cache Systems Engineer

Huawei Hong Kong Research Center(HKRC)

📍 Tai Po District, Hong Kong SAR, Hong Kong On-Site ⏳ 3h ago
AI assistant

Job Description

About the role

We are looking for an LLM KV Cache Systems Engineer to design and build KV Cache subsystems for large-language-model inference. You will work on memory allocation, lifecycle management, reuse, compression, transfer, tiering, and KV-aware scheduling across single-node and distributed serving systems.

This is a systems role. Direct source-level experience with an LLM KV Cache subsystem is more important than familiarity with serving-framework APIs alone.


What you will do

  • Design block/page allocators, KV Cache lifecycle management, fragmentation control, eviction, prefix caching, cross-request reuse, and preemption/recovery mechanisms.
  • Optimize KV layouts, GQA/MQA handling, quantization, compression, or sparsity while balancing accuracy, capacity, memory bandwidth, and compute overhead.
  • Build KV transfer and disaggregated-serving mechanisms, including prefill/decode