Generative AI services like ChatGPT, Claude, and Gemini show up in the news almost every day. At the center of these services, doing the actual reading and writing of text, is the LLM (Large Language Model). It is also a prime example of a concept many people have heard of but can’t quite explain.
This article lays out what an LLM is, how it produces human-like text, and what it is good and bad at—drawing on official documentation. Along the way, you’ll see how the concepts we’ve covered separately on this site—tokens, context windows, and hallucination—all sit on top of this single foundation called the LLM.
What Is an LLM: A “Model of Language” Trained on Vast Amounts of Text
Anthropic’s official glossary defines LLMs as “AI language models with many parameters that are capable of performing a variety of surprisingly useful tasks.” It adds that “these models are trained on vast amounts of text data and can generate human-like text, answer questions, summarize information, and more”1.
There are three points to note. First, LLMs are trained on enormous amounts of text. Second, they have a very large number of parameters (the internal adjustment dials of the model). Third, as a result, they can handle a wide range of tasks—text generation, question answering, summarization—without being purpose-built for any one of them. Whereas traditional AI was built per use case (“translation only,” “classification only”), the big difference with LLMs is that a single model can be applied to many different jobs.
Note that “generative AI” and “LLM” are often confused. Generative AI is the broader concept—it includes images, audio, and video—and the LLM is the core technology within it that handles language.
How It Works: Predicting the “Next Word,” One Token at a Time
The mechanism by which an LLM generates text rests on a surprisingly simple principle: repeatedly predicting the next word.
First, the input text is broken into units called tokens. According to Microsoft’s explanation, when you work with an LLM, “text is first broken into units called tokens, which are words, character sets, or combinations of words and punctuation, by a tokenizer”3. Then, during training, the LLM “analyzes the semantic relationships between tokens, such as how commonly they’re used together or whether they’re used in similar contexts”3.
These “semantic relationships” expressed as numbers are called embeddings. Microsoft’s explanation states: “These token ID sequences reveal the semantic relationships between tokens. Multi-valued numeric vectors, known as embeddings, represent these relationships. The model assigns an embedding to each token based on how commonly it’s used together with, or in similar contexts to, the other tokens”3. Put crudely, the LLM converts words into “coordinates” and captures the relationships between them by placing words with similar meanings close together.
When it generates text, it selects “the most probable next token to follow the sequence so far.” Microsoft’s explanation says that “during output generation, the model predicts a vector value for the next token in the sequence. The model then selects the next token from its vocabulary based on this vector value”3, and further that “output generation is an iterative operation. The model appends the predicted token to the sequence so far and uses that as the input for the next iteration, building the final output one token at a time”3. The way a chat AI’s answer appears bit by bit from the start is exactly this “one token at a time” in action.
What an LLM can reference is not the training data itself but the text that fits within the current exchange. This “working memory” is the context window, which Anthropic describes as “the amount of text a language model can look back on and reference when generating new text”—distinct from the large corpus it was trained on, and instead “a ‘working memory’ for the model”1.
Why It Suddenly Got Smart: The Transformer Turning Point
Behind how practical LLMs have become lies an architecture introduced in 2017: the Transformer.
The starting point was the paper “Attention Is All You Need”2, published in June 2017 by researchers at Google and others. The paper proposed a new architecture, the Transformer, that did not use the then-dominant sequential networks (recurrence or convolutions) but was instead “based solely on attention mechanisms”2.
In plain terms, the attention mechanism is “a way of deciding which words in a sentence to focus on by surveying the whole sentence.” Because it doesn’t need to process one word at a time sequentially, parallel computation is easy and training time can be greatly shortened. The paper reported achieving 28.4 BLEU on English-to-German and 41.8 BLEU on English-to-French machine translation—state of the art at the time—with substantially less training time2. This property of “learning efficiently from large amounts of data” opened the road to later scaling up: to “Large” models.
How a “Mere Prediction Model” Becomes an Assistant
Here a question arises. Why does a model that merely predicts the next word follow instructions and answer politely? The answer is that an LLM goes through multiple training stages.
The first stage is pretraining. Anthropic’s glossary defines pretraining as “the initial process of training language models on a large unlabeled corpus of text,” explaining that “in Claude’s case, autoregressive language models (like Claude’s underlying model) are pretrained to predict the next word, given the previous context of text in the document”1. But there’s an important caveat: “These pretrained models are not inherently good at answering questions or following instructions, and often require deep skill in prompt engineering to elicit desired behaviors”1.
That’s where fine-tuning and RLHF come in next. Anthropic defines fine-tuning as “the process of further training a pretrained language model using additional data”1. As for RLHF (Reinforcement Learning from Human Feedback), it is “a technique used to train a pretrained language model to behave in ways that are consistent with human preferences,” where “human feedback consists of ranking a set of two or more example texts, and the reinforcement learning process encourages the model to prefer outputs that are similar to the higher-ranked ones”1.
Indeed, Anthropic states that “Claude is not a bare language model; it has already been fine-tuned to be a helpful assistant”1, and that “Claude is a conversational assistant based on a large language model that has been fine-tuned and trained using RLHF to be more helpful, honest, and harmless”1. In other words, the chat AI we use every day is the result of layering learned, human-preferred behavior on top of a bare “next-word-predicting” LLM.
What It Can Do, and What It Struggles With
LLMs can handle a wide range of language tasks—text generation, question answering, summarization, translation, classification1. At the same time, the limitations rooted in how they work are clear.
The biggest weakness is hallucination. Because an LLM is, after all, probabilistically choosing “a plausible next word,” it can confidently output content that differs from the facts. The reason Anthropic lists “honest” as one of its ideals for AI—defining an honest AI as one that will “give accurate information, and not hallucinate or confabulate”1—is, conversely, that this is a structural challenge for LLMs.
Another limitation is the scope of knowledge. An LLM’s knowledge depends on its training-time data; it doesn’t know events after that point or your organization’s private information. A representative method for compensating for this weakness is RAG (retrieval-augmented generation), which passes external knowledge into the context at run time to ground the answer. Furthermore, technologies like AI agents—which let an LLM use tools and advance tasks autonomously—and MCP—which standardizes connections to external data and tools—were all created to extend the LLM’s capabilities, on the assumption of the LLM as the foundation.
The LLM is the simple principle of “predicting the next word,” grown into a remarkably general-purpose tool through vast data, the efficient learning enabled by the Transformer, and tuning toward human preferences. Once you grasp how it works and where its limits lie, it becomes much easier to judge how far to trust generative AI and where humans should fill in the gaps.
Sources
- Glossary - Anthropic’s official glossary (definitions of LLM, Pretraining, Fine-tuning, RLHF, Context window)
- Attention Is All You Need - Vaswani et al. (the original paper proposing the Transformer, 2017)
- Understanding tokens - Microsoft Learn official explanation (tokenization, embeddings, next-token prediction)