Question

A data science team is explaining to non-technical stakeholders why modern large language models can understand relationships between words that are far apart in a sentence, such as connecting a pronoun to a noun mentioned many words earlier. Which architectural feature of the transformer model is primarily responsible for this capability?

Accepted Answer

The self-attention mechanism, which weighs the relevance of all tokens in the sequence relative to one another

Answer

The convolutional filters that slide across fixed-size windows of the input text

Answer

The recurrent connections that process tokens strictly one at a time in order

Answer

The max-pooling layers that downsample the sequence to reduce its length