AI Inference & Serving
Mixed-Precision Inference
Running different parts of a model with different numerical precisions, such as lower-precision operations alongside selected higher-precision operations. It is not synonymous with converting every tensor to one smaller type. Conversions, accumulation rules and sensitive operations affect numerical behavior, so a mixed arrangement needs validation on the actual workload.
Reviewed
Sources
Free complete lesson
This definition and the complete practical lesson are free. The interactive reader loads the lesson examples without a membership.