Parameter
A learnable weight in a neural network; model size is measured in billions of parameters.
Parameters are the numerical values in a neural network that are learned during training and determine its behavior. Larger parameter counts generally correlate with greater capability but also higher inference cost. Models are commonly described by parameter count: 7B, 70B, 405B. However, MoE models have high total parameters but activate only a fraction per inference.
Verwandte Begriffe
Large Language Model — a neural network trained on vast text corpora to generate human-like text.
The neural network architecture underlying virtually all modern LLMs.
Compressing model weights to lower numerical precision to reduce memory and speed up inference.
An architecture where only a subset of model parameters is activated per token.