Large Language Models Explained
A large language model predicts tokens in sequence. Scale, diverse training data and transformer networks allow this simple training objective to produce remarkably flexible language capabilities.
Learning objectives
What you will be able to do
- Explain large language models explained accurately in your own words.
- Recognise how the concept appears in real AI products and professional work.
- Identify an important limitation, risk or evaluation requirement.
- Apply the concept through a practical activity and knowledge check.
Core explanation
Build the right mental model
During pretraining, the model repeatedly predicts missing or next tokens across very large text and code collections. The learned parameters capture statistical relationships among words, concepts, styles and structures.
Instruction tuning and preference-based methods make the pretrained model more useful in conversation. Retrieval, tools and system instructions can further adapt behaviour without changing all model parameters.
An LLM has a knowledge cutoff, finite context and no automatic guarantee of truth. It may combine patterns into a convincing but unsupported claim. Use retrieval and verification for consequential work.
Pretraining
Broad learning from large datasets using a prediction objective.
Instruction tuning
Additional training that improves response to user instructions.
Inference
Using the trained model to generate an output.
Hallucination
A plausible output not adequately supported by facts or provided evidence.
Technical support assistant
An LLM explains an error and proposes troubleshooting steps.
Apply your learning
Practical activity
- Ask an LLM about a topic you know well.
- Mark statements as supported, uncertain or wrong.
- Repeat with authoritative source text supplied in the prompt and compare.
Write your answers in a learning journal. The value comes from connecting the concept to your own profession.
Knowledge check
Test your understanding
Q1What is the basic pretraining objective of many LLMs?
Answer: Predicting tokens based on surrounding or preceding context.
Q2Why can an LLM hallucinate?
Answer: It optimises plausible token generation rather than directly checking every statement against an authoritative fact source.
If you cannot explain the answer without reading it, revisit the core explanation and example before continuing.
Continue with authoritative sources
Important industry resources
These links lead to official organisations, industry laboratories or established open-source learning projects.