🎓 Free video course "LLM evaluation for AI builders" with 10 code tutorials.  Save your seat
LEARN

AI quality and MLOps tutorials

Open-source tutorials on AI and ML observability, LLM evaluation, and MLOps for data scientists and engineers.
All tutorials contain end-to-end code examples.
Wrong but useful: an LLM-as-a-judge tutorial
Tutorials

Wrong but useful: an LLM-as-a-judge tutorial

This tutorial shows how to create, tune, and use LLM judges. We'll make a toy dataset and assess correctness and verbosity. You can apply the same workflow for other criteria.
LLM regression testing workflow step by step: code tutorial
Tutorials

LLM regression testing workflow step by step: code tutorial

In this tutorial, we introduce the end-to-end workflow of LLM regression testing. You will learn how to run regression testing as a process and build a dashboard to monitor the test results.
A tutorial on regression testing for LLMs
Tutorials

Watch the language: A tutorial on regression testing for LLMs

In this tutorial, you will learn how to systematically check the quality of LLM outputs. You will work with issues like changes in answer content, length, or tone, and see which methods can detect them.
Batch inference and ML monitoring with Evidently and Prefect
Tutorials

Batch inference and ML monitoring with Evidently and Prefect

In this tutorial, you will learn how to run batch ML model inference and deploy a model monitoring dashboard for production ML models using open-source tools.
How to start with ML model monitoring. A step-by-step guide.
Tutorials

How to stop worrying and start monitoring your ML models: a step-by-step guide

A beginner-friendly MLOps tutorial on how to evaluate ML data quality, data drift, model performance in production, and track them all over time using open-source tools.
Monitoring unstructured data for LLM and NLP with text descriptors
Tutorials

Monitoring unstructured data for LLM and NLP with text descriptors

How do you monitor unstructured text data? In this code tutorial, we’ll explore how to track interpretable text descriptors that help assign specific properties to every text.
A simple way to create ML Model Cards in Python
Tutorials

A simple way to create ML Model Cards in Python

In this code tutorial, you will learn how to create interactive visual ML model cards to document your models and data using Evidently, an open-source Python library.
ML serving and monitoring with FastAPI and Evidently
Tutorials

ML serving and monitoring with FastAPI and Evidently

In this code tutorial, you will learn how to set up an ML monitoring system for models deployed with FastAPI. This is a complete deployment blueprint for ML serving and monitoring using open-source tools.
Batch ML monitoring blueprint: Evidently, Prefect, PostgreSQL, and Grafana
Tutorials

Batch ML monitoring blueprint: Evidently, Prefect, PostgreSQL, and Grafana

In this code tutorial, you will learn how to run batch ML model inference, collect data and ML model quality monitoring metrics, and visualize them on a live dashboard.
How to set up ML monitoring with email alerts using Evidently and AWS SES
Tutorials

How to set up ML monitoring with email alerts using Evidently and AWS SES

In this tutorial, you will learn how to implement Evidently checks as part of an ML pipeline and send email notifications based on a defined condition.
A tutorial on building ML and data monitoring dashboards with Evidently and Streamlit
Tutorials

A tutorial on building ML and data monitoring dashboards with Evidently and Streamlit

In this tutorial, you will learn how to create a data quality and ML model monitoring dashboard using the two open-source libraries: Evidently and Streamlit.
Monitoring NLP models in production: a tutorial on detecting drift in text data
Tutorials

Monitoring NLP models in production: a tutorial on detecting drift in text data

In this tutorial, we will explore issues affecting the performance of NLP models in production, imitate them on an example toy dataset, and show how to monitor and debug them.
How to set up ML Monitoring with Evidently. A tutorial from CS 329S: Machine Learning Systems Design.
Tutorials

How to set up ML Monitoring with Evidently. A tutorial from CS 329S: Machine Learning Systems Design.

Our CTO Emeli Dral gave a tutorial on how to use Evidently at the Stanford Winter 2022 course CS 329S on Machine Learning System design. Here is the written version of the tutorial and a code example.
How to detect, evaluate and visualize historical drifts in the data
Tutorials

How to detect, evaluate and visualize historical drifts in the data

You can look at historical drift in data to understand how your data changes and choose the monitoring thresholds. Here is an example with Evidently, Plotly, Mlflow, and some Python code.
What Is Your Model Hiding? A Tutorial on Evaluating ML Models
Tutorials

What Is Your Model Hiding? A Tutorial on Evaluating ML Models

There is more to performance than accuracy. In this tutorial, we explore how to evaluate the behavior of a classification model before production use.
How to break a model in 20 days. A tutorial on production model analytics
Tutorials

How to break a model in 20 days. A tutorial on production model analytics

What can go wrong with ML model in production? Here is a story of how we trained a model, simulated deployment, and analyzed its gradual decay.
Learn

Want more than a tutorial?
Take a course!

LLM evaluations for AI builders

Sign up for our free video course with 10 hands-on code tutorials. From designing custom LLM judges to RAG evaluations and adversarial testing. If you’re working hands-on with LLMs and have basic Python skills, this course is for you.
Sign up now
Icon

LLM evaluations for AI product teams

Building an LLM-powered product? Sign up for our free email course on LLM evaluations for AI product teams. A gentle introduction to evaluating LLM-powered apps, no coding knowledge required.
Sign up now
Icon

Open-source ML observability course

Want to learn production ML monitoring? Sign up for our Open-source ML observability course for data scientists and ML engineers. It's free!
Start now
Icon
Evidently Getting Started tutorial

Start testing your AI systems today

Book a personalized 1:1 demo with our team or sign up for a free account.
Get demo
Icon
No credit card required
© 2025, Evidently AI. All rights reserved
Twitter logoDiscord logoYouTube logo
By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
DenyAccept