AI Inference & Serving
Speculative Decoding
An acceleration approach that proposes token continuations and verifies them with a target model. Correct algorithms can preserve the target sampling distribution; proposal, verification and implementation details determine the guarantee.
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.