Section 13 of 24 · 4 lessons

Adapting and Evaluating Modern Models

A trained model may be capable of many things without doing the particular job we need. We can teach it from examples, give it feedback, or provide information it did not have during training. We also need to check whether those changes actually make it more useful.

0 of 4 finished

Adapting a Trained Model

We may already have a useful model but need it to respond differently for a particular task. Examples, smaller parameter updates, and rewards offer different ways to teach that behaviour.

  1. 01Supervised Fine-TuningA model that can continue text may not respond in the way a task requires. Fine-tuning on examples of prompts and desired responses gives it demonstrations of the behaviour we want.
    Build with Botie →Read the lesson
  2. 02Low-Rank AdaptationAdapting every weight in a large model can be expensive. LoRA keeps the original weights fixed and learns a smaller correction through two narrow matrices, reducing the number of parameters that need updating.
    Build with Botie →Read the lesson

Grounding and Evaluation

An answer can sound convincing and still be wrong. We need to provide relevant information when the task requires it and evaluate the answer against what the user actually needed.

  1. 03Retrieval-Augmented GenerationA model may need information that was absent or out of date when it was trained. Retrieval supplies relevant source material as context, giving the model evidence it can use when answering a question.
    Build with Botie →Read the lesson
  2. 04Evaluating Generative ModelsA fluent answer or convincing picture is not enough to show that a model does its job well. We need evaluation examples and criteria that reflect the task, including failures an overall average might conceal.
    Build with Botie →Read the lesson