Architecture
Transformer
The neural network architecture underlying virtually all modern LLMs.
Introduced in 'Attention Is All You Need' (2017), the Transformer replaced recurrent networks with self-attention layers that process all tokens in parallel. This enabled efficient training at scale and is the foundation of GPT, BERT, Claude, Gemini, and virtually every production LLM in use today.
Termes Associés
Attention Mechanism
The core operation in transformers that lets each token attend to all other tokens.
LLM (Grand Modèle de Langage)
Un réseau de neurones entraîné sur de grands volumes de texte pour générer du contenu.
Parameter
A learnable weight in a neural network; model size is measured in billions of parameters.
Pre-Training
The initial large-scale training phase where a model learns language from massive text corpora.