AI Inference & Serving
Time to First Token
The elapsed time from a specified request start to its first output token. State the measurement boundary: client-observed time can include network and queue delays that an internal model timer excludes.
Also called: TTFT
Reviewed
Sources
Free complete lesson
This definition and the complete practical lesson are free. The interactive reader loads the lesson examples without a membership.