Deployment & Scaling · Advanced
Evaluation, Prompt Flow and Observability on Foundry
How to know whether your Azure AI application is any good - golden sets, built-in evaluators, tracing, and continuous evaluation on live traffic.
- 55 min lesson
- advanced level
- Azure AI Track
- Narrated presentation
What you’ll learn
- A frozen, versioned golden set of 50-200 real examples as the unit of measurement
- Evaluate retrieval and generation separately, always
- Quality, risk-and-safety, and agent evaluator families, plus custom evaluators
- Model-graded evaluators need calibration against human labels before being quoted
- Prompt flow concepts: flows, nodes, variants and batch runs
- Tracing to Application Insights and continuous evaluation on sampled production traffic
- Generative quality regressions are silent - no error rate rises
Inside this lesson
A narrated, slide-by-slide presentation with 1 prediction exercise, 1 knowledge check, 3 videos. You answer as you go, and the platform checks your work.