← All learning areas

AI Agents and Applications

A model answering a question may need to look up a record or ask another program to do something. Build an application that can handle those requests, then test where it makes mistakes.

Agents
A technician asks Botie about a pump, but the answer needs a manual held in another system. How can a model request that information, and who decides what it may do with it? Start with the pieces of an agent, then build the maintenance assistant using Pydantic AI, LangChain and LangGraph. As the job grows, add permission checks and guardrails, and handle calls whose replies never arrive.18 lessons

Prompting, Evaluation and Iteration
Botie answers the first maintenance question correctly, then gets the next one wrong. Before rewriting the prompt, we need to understand what happened. Choose useful examples, follow the tool calls through their traces, and build tests that expose the failure. We will use Langfuse and other evaluation tools, compare candidate revisions, and carry a repair through a fresh shift without losing the behaviours that already worked.12 lessons

Agent Guardrails and Secure Development
Botie can now write code, call MCP tools and change application records. What should he be unable to do, even if a document misleads him or a proposed fix is wrong? Build the boundaries around those effects. We will inspect patches, restrict execution, check delegated authority and follow reviewed actions through retries, then test the repaired system against a different shift.9 lessons