🎓 Free video course "LLM evaluation for AI builders" with 10 code tutorials.  Save your seat

Open-source AI observability course

Free Open-source AI observability course for data scientists and ML engineers. Join us to learn AI observability production ML monitoring.

Explore course notes
Icon
Evidently AI Open-source ML observability course
about the course

What you will learn

The foundations of ML observability: from exploratory data analysis and model evaluation pre-deployment to continuous production monitoring and debugging.
Icon
How to evaluate ML models and design ML monitoring strategy.
Icon
How to monitor ML models for batch and real-time systems.
Icon
How to test data quality in production ML pipelines.
Icon
How to monitor quality of NLP and LLM systems.
Icon
How to detect and interpret data and prediction drift.
Icon
How to debug model performance decay.
perks

Why take the course?

Join the course to learn production ML monitoring.
Icon
Free
Icon
40+ on-demand videos
Icon
Practical code examples

Get notified about new courses

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
intro video

Welcome to the course

testimonials

What the students say

1200+ students have signed up to take the course.
Stamatis Karlos
Stamatis Karlos
Data Scientist, efood
"Model drift is not a phenomenon that will be eliminated from one day to another in production level. But after that course, someone can be ready to act before critical performance/business issues are raised."
Nissim Matatov
Nissim Matatov
Data Scientist and ML Engineer
"The course covered a wealth of practical MLOps material and featured many insightful discussions led by the course team. I highly recommend incorporating Evidently AI's open-source methodology and products into your MLOps toolbox."
Stamatis Karlos
Stamatis Karlos
Data Scientist, efood
"Model drift is not a phenomenon that will be eliminated from one day to another in production level. But after that course, someone can be ready to act before critical performance/business issues are raised."
Nissim Matatov
Nissim Matatov
Data Scientist and ML Engineer
"The course covered a wealth of practical MLOps material and featured many insightful discussions led by the course team. I highly recommend incorporating Evidently AI's open-source methodology and products into your MLOps toolbox."
Vishal Kumar S.
Vishal Kumar S.
Data Science and ML Enthusiast
"I enjoyed the Open-source ML Observability course, gaining valuable insights into the importance of monitoring throughout the ML pipeline. The practical integration of Evidently with tools like MLflow and Airflow enriched my learning experience, enhancing my skills in ML observability."
Victor Matekole
Victor Matekole
ML Engineer
"The course equipped me to navigate the intricate relationship between machine learning and business goals, giving me a holistic approach to MLOps. Emeli Dral is a great teacher, highly recommended."
Vishal Kumar S.
Vishal Kumar S.
Data Science and ML Enthusiast
"I enjoyed the Open-source ML Observability course, gaining valuable insights into the importance of monitoring throughout the ML pipeline. The practical integration of Evidently with tools like MLflow and Airflow enriched my learning experience, enhancing my skills in ML observability."
Victor Matekole
Victor Matekole
ML Engineer
"The course equipped me to navigate the intricate relationship between machine learning and business goals, giving me a holistic approach to MLOps. Emeli Dral is a great teacher, highly recommended."
modules

Course syllabus

The course is organized into six modules. You can follow the complete course syllabus or pick only the modules that are most relevant to you.
Introduction to ML monitoring and observability

Introduction to ML monitoring and observability

Basics of ML monitoring and observability.
ML monitoring metrics: model quality, data quality, data drift

ML monitoring metrics: model quality, data quality, data drift

Methods and evaluation metrics for structured data.
Monitoring unstructured data: LLM, NLP, and embeddings

Monitoring unstructured data: LLM, NLP, and embeddings

Methods and evaluation metrics for unstructured data.
Designing effective ML monitoring

Designing effective ML monitoring

Customizing ML monitoring to your model.
ML pipelines validation and testing

ML pipelines validation and testing

Deploying an end-to-end pipeline to check data and ML model quality.  
Deploying an ML monitoring dashboard

Deploying an ML monitoring dashboard

How to deploy an ML monitoring service and design a live dashboard.  
Icon
Icon
Start now
Icon
LEARN now

Course materials

We published all 40 lessons with videos, course notes, and code examples so you can learn at your own pace.
Jump to the course website to start learning now!
Evidently AI Open-source ML observability courseExplore course notes
team

Meet course authors

Emeli Dral
LinkedIn logo

Emeli Dral

Co-founder and CTO
Evidently AI

Emeli Dral is a Co-founder and CTO at Evidently AI, a startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.
‍
Earlier, she co-founded an industrial AI startup and served as the Chief Data Scientist at Yandex Data Factory. She led over 50 applied ML projects for various industries: from banking to manufacturing. Emeli is a data science lecturer at GSOM SpBU and Harbour.Space University. She is a co-author of the Machine Learning and Data Analysis curriculum at Coursera with over 150,000 students.

Elena Samuylova is a CEO and Co-founder at Evidently AI, a Y Combinator-backed startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.

‍

She has been active in the applied machine learning space since 2014. Previously, she co-founded and served as a CPO of an industrial AI startup. She worked with global metal and chemical companies to implement machine learning for production optimization. Prior to that, she led business development at Yandex Data Factory, an enterprise AI division of Yandex. She focused on delivering ML-based solutions to retail, banking, telecom, and other industries. In 2018, Elena was named 50 Women in Product Europe by Product Management Festival.

Faq

Have a question or need help?

Is the course free?
Icon

Yes, the course is 100% free.

What is the course format?
Icon

The course is organized into six modules. Each module consists of on-demand videos and practical code examples. Learn at your own pace with publicly available course materials.

Are there any course prerequisites?
Icon

There are both theoretical and code-focused modules that require knowledge of Python. But no worries: we will walk you through the code! You can also skip these parts and still learn a lot.

Where can I find course materials?
Icon

We published all 40 lessons with videos, course notes, and code examples. Jump to the course website to start learning now!

Is there a newsletter?
Icon

Yes, sign up here to be notified about new courses.

Evaluate, test and monitor your AI-powered products.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
LLM testing platformAI risk assessmentRAG testingAdversarial testingAI agent testingML monitoringOpen-source
BlogTutorialsGuidesML platformsML use casesLLM evaluations course
PricingDocsGitHubCommunity
© 2025, Evidently AI. All rights reserved.
Twitter logoDiscord logoYouTube logo
By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.
DenyAccept