What Are LLM Parameters? Understanding the '7B' and '70B' in AI Model Sizes

AI model names often carry a number and the letter “B,” as in “Llama 3.1 70B” or “Mixtral 8x7B.” This B is a unit expressing a model’s size—its parameter count. It is the first number you see whether choosing a model or reading the news, yet its meaning is surprisingly seldom explained.

This article explains, for non-specialists and drawing on the scaling-laws paper and technical sources, what parameter count is, why more tends to mean smarter, and the reasons you can’t simply say “bigger is better.” Reading it alongside the basics of LLMs will deepen your understanding.

Parameters Are “Numbers That Memorize What Was Learned”

A parameter, to borrow a technical explainer’s words, is “an internal numerical value that holds the knowledge the model has learned and is used to generate responses” ². As a model learns from vast amounts of text, it gradually adjusts these values, and it is said to “memorize, as numbers, the relationships between words and sentences, grammar rules, and even facts about the world” ².

To use a brain analogy, it is close to the strength of connections between neurons. Each parameter is just a small number, but when billions of them combine, the complex ability to understand and generate text emerges.

”B” Means Billion—7B Is 7 Billion

The “B” in a model name is short for Billion ³. So “7B” means about 7 billion parameters, and “70B” means about 70 billion.

For example, Llama 3.1 comes in versions of different sizes: “8 billion (8B), 70 billion (70B), and 405 billion (405B)” ². Even within the same model family, multiple sizes are offered so you can choose according to your use case and the environment you’ll run it in.

More Tends to Be Smarter: Scaling Laws

So what is the benefit of having more parameters? Basically, more tends to bring higher performance. The empirical rule that backs this up theoretically is called the scaling laws.

A technical explainer describes the scaling laws as “an empirical rule that as you increase model size (parameter count), dataset size, and the compute used for training, the model’s performance improves in a predictable way” ². This was shown in OpenAI’s 2020 paper “Scaling Laws for Neural Language Models” (Kaplan et al.), which reported that loss (the magnitude of the model’s error) “shows a power-law relationship with model size, dataset size, and the amount of compute used for training” ¹. Moreover, this trend is said to hold over a very wide range of more than seven orders of magnitude ¹, and it became the basis for the recent race to scale up models.

Why You Can’t Simply Say “Bigger Is Better”

That said, parameter count alone does not determine how smart a model is. The scaling laws say performance rises “if you increase size, data, and compute together,” and even at the same parameter count, performance varies with the quality of training data and the training method. In fact, even relatively small models are known to approach large ones through good training or fine-tuning.

An even bigger issue is the cost of running them. The more parameters, the more memory and the more powerful a GPU you need to run the model. A technical explainer notes that “the larger the parameter count, the longer the inference time and the more powerful a machine you need” ³. Conversely, a benefit of small models (SLMs) is that “the computational resources (such as GPUs) needed for training and operation are smaller” ². Intelligence and ease of running are in a trade-off.

How Much Memory Do You Need?

The amount of memory needed depends not only on parameter count but also on how many bits of precision each parameter is stored in. By a technical explainer’s estimate, with the common FP16 format (16-bit) the rule of thumb is “about 2GB per 1B,” and with coarser INT4 (4-bit quantization) it is “about 0.5GB per 1B” ³.

Concretely, an 8B Llama-3 runs in about 16GB at FP16 and about 4GB with INT4 quantization, while a 175B GPT-3 is estimated to need about 350GB at FP16 and still about 90GB with INT4 ³. Whether you can run it on your own PC often comes down to this memory figure.

This is where model compression (quantization) comes in. By lowering numerical precision, you can greatly reduce the memory needed even for a model with the same parameter count. As in the example above, using INT4 quantization brings the required memory to about a quarter of FP16, making it possible to run large models in more modest environments.

The “7B” or “70B” in a model name is not a mere brag about scale; it is the first clue for reading off “how smart it tends to be, and how much of an environment it takes to run.” When you come across news of a new model, viewing this number from both the “intelligence” and “ease of running” angles helps reveal the position that model is aiming for. If a token is the unit for measuring “the amount of text being processed,” it helps to think of parameter count as the unit for measuring “the size of the model itself.”