AI Inference & Serving · Emerging terminology
Attention Sink
A token position that attracts substantial attention even when its semantic content does not explain that attention. Research on streaming language models found that retaining certain initial positions can stabilize particular caching arrangements. This observed behavior is not evidence that the token is an important fact or a permanent memory of everything before it.
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.