Large language models (LLMs) power today's chatbots and writing assistants. Their abilities can seem mysterious, but the core idea is surprisingly compact.

Predicting the next token

Text is broken into small pieces called tokens — whole words or fragments of words. During training, the model sees enormous amounts of text and learns to predict which token is likely to come next. Generating an answer means repeating that prediction, one token at a time.

The transformer

Most modern LLMs are built on the transformer architecture, introduced in the 2017 paper Attention Is All You Need. Its key mechanism, attention, lets the model weigh how relevant every earlier token is when predicting the next one.

Strengths and limits

  • LLMs can summarise, translate, explain and write code with impressive fluency.
  • They can also produce confident statements that are false — often called hallucinations.
  • Their knowledge comes from training data, so it can be incomplete or out of date.