Section 15 of 24 · 6 lessons

Reinforcement Learning

Botie sometimes has to act before he knows which choice is best. We will begin with the feedback an action produces, then add changing states and rewards that arrive later. Planning with a known model and learning from observed transitions will give us two ways to improve his decisions.

0 of 6 finished

Learning From Consequences

Botie sees feedback only after choosing an action. We will distinguish learning about independent choices from decisions that change the state he faces next.

  1. 01Reinforcement LearningWe cannot always provide the correct answer before an action is taken. Reinforcement learning uses the rewards that follow actions to improve a policy, including decisions whose consequences appear several steps later.
    Build with Botie →Read the lesson
  2. 02Trying an Action Before Knowing Its ValueBotie can choose a watering schedule, but sees the result only for the schedule he actually tries. He must balance using his current favourite with learning about alternatives.
    Build with Botie →Read the lesson
  3. 03Describing What an Action ChangesWatering changes the greenhouse Botie will face tomorrow. A state, an action, and a transition model let us describe that consequence before choosing a policy.
    Build with Botie →Read the lesson

Planning Beyond the Next Reward

An action can pay off later. Keep the reward sequence visible as we calculate returns, plan with a transition model, and learn action values from experience.

  1. 04Valuing Rewards That Arrive LaterAn action can cost water now and help a plant later. A return combines a sequence of rewards so Botie can compare more than the immediate outcome.
    Build with Botie →Read the lesson
  2. 05Planning One Step, Then the Remaining JourneyA long sequence of choices is difficult to compare directly. We can combine the reward from one action with the value of the state it leaves behind, then improve those estimates repeatedly.
    Build with Botie →Read the lesson
  3. 06Learning Action Values From ExperienceBotie may not know the transition probabilities. A sampled action, reward, and next state can still correct one entry in an action-value table.
    Build with Botie →Read the lesson