Reinforcement Learning Fundamentals
Reinforcement learning trains an agent through interaction. Actions influence an environment, rewards communicate outcomes, and the agent learns a policy for choosing future actions.
Learning objectives
What you will be able to do
- Explain reinforcement learning fundamentals accurately in your own words.
- Recognise how the concept appears in real AI products and professional work.
- Identify an important limitation, risk or evaluation requirement.
- Apply the concept through a practical activity and knowledge check.
Core explanation
Build the right mental model
Unlike supervised learning, the agent is not shown the correct action at every step. It explores, observes consequences and attempts to maximise cumulative reward. This makes reinforcement learning useful for sequential decisions.
Reward design is difficult. A poorly specified reward can produce behaviour that technically scores well while violating the designer’s intention. Safe exploration is also essential when actions can affect people, money or physical systems.
Reinforcement learning powers research in games, robotics, resource allocation and control. Human-feedback techniques also use preference signals to shape generative-model behaviour.
Describe a sequential decision problem
Reinforcement learning studies an agent that selects actions, observes consequences and receives rewards. A policy describes how it chooses actions from available information. The environment determines what happens next. Unlike a list of independently labelled examples, actions can change the situations the agent encounters later.
For a warehouse robot, moving toward a shelf changes its position and may affect the route to the next shelf. For a toy learning exercise, however, use a paper grid or a simulation. Exploring actions on real equipment can create hazards and should not be treated as an ordinary classroom experiment.
Distinguish immediate reward from return
A reward is a signal associated with a step or transition. Return combines rewards over a sequence, sometimes discounting later rewards by a factor gamma between zero and one. An agent seeking high return may accept a short-term cost to reach a better later outcome. This is why choosing the action with the largest immediate reward need not be the best policy.
Exploration tries actions to learn about their consequences. Exploitation uses currently preferred actions. Too little exploration can leave a better option undiscovered; unrestricted exploration can waste resources or cause harm. In a safe paper exercise, you can inspect every route. A real system needs explicit limits on what experimentation is permitted.
Test whether the reward matches the purpose
A reward function encodes what receives numerical credit, not everything humans care about. If an agent receives points each time it reports a completed job, it might find a way to report completion without doing useful work unless the environment and checks prevent that. The problem is a mismatch between the measured objective and the intended outcome.
Include constraints and evaluation cases that reveal undesirable shortcuts. For a delivery task, reaching the destination matters, but so do collisions, restricted areas and energy use. Adding a penalty can help express a trade-off, yet it is not a proof of safety. Some actions should be prohibited by the system rather than merely assigned a negative reward.
Agent
The decision-maker that selects actions.
Environment
The world the agent observes and influences.
Reward
A numerical signal indicating immediate outcome quality.
Policy
The strategy used to choose an action from a state.
Compare two toy routes by discounted return
A fictional game offers two routes. Route A gives rewards 0 now, 0 next and 10 on the third step. Route B gives 4 now and 4 on the second step. With gamma = 0.9, Route A’s return is 0 + 0.9 × 0 + 0.9² × 10 = 8.1. Route B’s return is 4 + 0.9 × 4 = 7.6. Under these assumptions, A has the larger return even though its first reward is smaller.
With gamma = 0.5, A gives 2.5 and B gives 6. The preference changes because later rewards receive less weight. This arithmetic does not say which discount factor is ethically or operationally correct. It shows that the objective definition affects the preferred behaviour.
If A passes through a prohibited area, its higher numerical return should not make it allowable. A policy must operate within the task’s constraints. Keep the reward calculation and the permission question separate when designing a real workflow.
Apply your learning
Practical activity
- Spend five minutes drawing a safe paper grid with a start, goal and prohibited square.
- Spend five minutes listing two allowable routes and assigning invented rewards to each step.
- Spend five minutes calculating each route’s return using gamma = 1 and gamma = 0.8.
- Spend five minutes identifying a way to collect rewards without meeting the intended goal, then revise the task rules.
Compare with an explained example answer
For a route with rewards −1, −1 and 8, undiscounted return is 6. At gamma = 0.8 it is −1 − 0.8 + 0.64 × 8 = 3.32. Showing each term makes the timing convention visible.
A repeated “visit checkpoint” bonus could encourage looping instead of finishing. One repair is to award it only once per episode and require a verified goal state. Explain which actions are disallowed outright; a large reward elsewhere should not override a prohibited move.
Write your answers in a learning journal. The value comes from connecting the concept to your own profession.
Knowledge check
Test your understanding
Q1What is the difference between reward and return?
Answer: Reward is a step-level signal; return combines rewards over a sequence, potentially discounting later values.
Q2Why might a greedy immediate-reward choice be poor?
Answer: It can sacrifice a larger future outcome. Route A in the example has a better return at gamma 0.9 despite no immediate reward.
Q3Does a carefully chosen reward guarantee safety?
Answer: No. Constraints, environment design, evaluation and oversight are also needed; an objective can omit important consequences.
If you cannot explain the answer without reading it, revisit the core explanation and example before continuing.
Continue with authoritative sources
Important industry resources
These links lead to official organisations, industry laboratories or established open-source learning projects.