AI Inference & Serving
Prefill-Decode Disaggregation
Serving prompt processing and autoregressive token generation on separate resource groups rather than necessarily keeping both phases on the same worker. The phases can be provisioned differently, but the required request state must cross the boundary. This is a serving architecture, not a different language-generation objective or a guarantee of lower latency.
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.