AI Inference & Serving
Data Parallel Inference
Distributing independent inference requests across model replicas or replica groups. This can increase serving capacity; it does not mean one request automatically uses all replicas or finishes sooner.
Also called: Request-Level Replication
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.