Concepts
Attention Mechanism
A model computation that combines information using context-dependent weights. In common query-key-value attention, scores between queries and keys determine how values are mixed. This is mathematical routing of information, not human concentration or a guaranteed explanation of why an entire model produced a particular answer.
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.