Parameter
A learnable weight in a neural network; model size is measured in billions of parameters.
Parameters are the numerical values in a neural network that are learned during training and determine its behavior. Larger parameter counts generally correlate with greater capability but also higher inference cost. Models are commonly described by parameter count: 7B, 70B, 405B. However, MoE models have high total parameters but activate only a fraction per inference.
Verwandte Begriffe
Büyük miktarda metin verisiyle eğitilmiş, insan benzeri metin üretebilen yapay sinir ağı.
The neural network architecture underlying virtually all modern LLMs.
Compressing model weights to lower numerical precision to reduce memory and speed up inference.
An architecture where only a subset of model parameters is activated per token.