Give us a star ⭐️ on GitHub to support the project!
🚀 Join us January 25 for the Evidently monthly demo and Q&A. Register now →

Open-source ML observability course 📈

Free Open-source ML observability course for data scientists and ML engineers. Join us to learn production ML monitoring.

The 2023 cohort has completed. You can learn at your own pace or sign up for the 2024 cohort.

Evidently Hacktoberfest 2022

What you will learn

The foundations of ML observability: from exploratory data analysis and model evaluation pre-deployment to continuous production monitoring and debugging.

  • How to evaluate ML models and design ML monitoring strategy.
  • How to test data quality in production ML pipelines.
  • How to detect and interpret data and prediction drift.
  • How to continuously monitor ML models for batch and real-time systems.
  • How to monitor changes in text data, and quality of NLP and LLM systems.
  • How to debug model performance decay.
  • Different ML monitoring architectures.
Open-sourse ML observability course
About the course:
✅ Free
🎥 6 modules with 40 on-demand videos
💻 Practical code examples
⭐️ Certificate upon completion

Sign up to join the 2024 cohort

The 2023 cohort of the Open-source ML observability course has completed.
Sign up to be notified when the 2024 cohort starts.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Welcome to the course

What the students say

Dayle Fernandes

Stamatis Karlos

Data Scientist, efood

“Model drift is not a phenomenon that will be eliminated from one day to another in production level. But after that course, someone can be ready to act before critical performance/business issues are raised.”

Moe Alter

Nissim Matatov

Data Scientist and ML Engineer

“The course covered a wealth of practical MLOps material and featured many insightful discussions led by the course team. I highly recommend incorporating Evidently AI's open-source methodology and products into your MLOps toolbox.”

Javier López Peña

Vishal Kumar S.

Data Science and ML Enthusiast

“I enjoyed the Open-source ML Observability course, gaining valuable insights into the importance of monitoring throughout the ML pipeline. The practical integration of Evidently with tools like MLflow and Airflow enriched my learning experience, enhancing my skills in ML observability.”

Javier López Peña

Victor Matekole

ML Engineer

“The course equipped me to navigate the intricate relationship between machine learning and business goals, giving me a holistic approach to MLOps. Emeli Dral is a great teacher, highly recommended.”

Course syllabus

Our course is organized into six modules. You can follow the complete course syllabus or pick only the modules that are most relevant to you.

SIGN UP for free

Course materials

We published all 40 lessons with videos, course notes, and code examples so you can learn at your own pace. Jump to the course website to start learning now!  

Open-source ML observability course from Evidently AI

Meet course authors

Emeli Dral

Emeli Dral

Co-founder and CTO
Evidently AI
Emeli Dral LinkedIn profile

Emeli Dral is a Co-founder and CTO at Evidently AI, a startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.

Earlier, she co-founded an industrial AI startup and served as the Chief Data Scientist at Yandex Data Factory. She led over 50 applied ML projects for various industries: from banking to manufacturing. Emeli is a data science lecturer at GSOM SpBU and Harbour.Space University. She is a co-author of the Machine Learning and Data Analysis curriculum at Coursera with over 150,000 students.

Elena Samuylova

Elena Samuylova

Co-founder and CEO
Evidently AI
Elena Samuylova LinkedIn profile

Elena Samuylova is a CEO and Co-founder at Evidently AI, a Y Combinator-backed startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.

She has been active in the applied machine learning space since 2014. Previously, she co-founded and served as a CPO of an industrial AI startup. She worked with global metal and chemical companies to implement machine learning for production optimization. Prior to that, she led business development at Yandex Data Factory, an enterprise AI division of Yandex. She focused on delivering ML-based solutions to retail, banking, telecom, and other industries. In 2018, Elena was named 50 Women in Product Europe by Product Management Festival.


Is the course free?

Yes, the course is 100% free.

What is the course format?

The course is organized into six modules. Each module consists of on-demand videos and practical code examples. 

To earn a certificate of completion, you must enroll in the course cohort. Sign up to save your seat and be notified when the next cohort is ready to launch.  

This is optional — you can also learn at your own pace and go through the course materials, which are publicly available.

How long does it take to complete the course?

The course lasts seven weeks. There are six modules with learning materials, and one extra week to complete the final assignment.

However, you can also learn at your own pace — whatever works best for you!

Will I get a course certificate?

You will receive a certificate if you enroll in the course cohort and successfully complete all the assignments.

Note that the option to receive the certificate is available only to those who participate in the course cohort. The next cohort will take place in 2024. Sign up to get updates when it starts.

Are there any course prerequisites?

There are both theoretical and code-focused modules that require knowledge of Python. But no worries: we will walk you through the code! You can also skip these parts and still learn a lot.

What if I need help?

Have a question or just want to say “Hi”? Jump to our Discord #-ml-observability-course channel to chat with fellow learners and get support from the course team. 

Where can I find course materials?

We published all 40 lessons with videos, course notes, and code examples. Jump to the course website to start learning now!

Is there a newsletter?

Yes, sign up here to receive course updates and be notified when the next cohort is ready to launch.

Will course materials still be available after the course?

Yes! All course materials are public so you can get back to them at your convenience, during or after the course. 

How to participate?

In the Evidently GitHub repository, we added a special set of issues labeled “hacktoberfest."

step 1
Check out the guide

Head to Evidently Hacktoberfest guide for clear steps and detailed examples.

step 2
Pick an issue

Check out the issues we prepared for Hacktoberfest. You can pick one of them or propose a different metric.

step 3
Submit pull request

Choose the drift method you want to implement and submit your pull request!

step 4
Get feedback

Wait for your pull request to be reviewed.

What is Evidently?

Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test, and monitor the performance of ML models from validation to production. You can check it out on GitHub or explore the documentation.

What is Hacktoberfest?

Hacktoberfest is an annual event to celebrate open-source and encourage contributions. It runs for the 9th time this year. Eligible participants will get prizes from DigitalOcean. But first and foremost, it is a great reason to create your first (or hundredth) pull request! You can check out the complete rules here.

How can I contribute?

Evidently is an open-source project, and is always open for contributions. For Hacktoberfest, we added a special set of issues labeled “hacktoberfest” to the Evidently GitHub repository. We invite data scientists to dip their toes into open-source contribution and help us add new statistical metrics and tests to detect data drift for production ML models. Check out Hacktoberfest issues on our GitHub. Head to the Evidently Hacktoberfest guide for clear steps and detailed examples. Sign up to receive the kick-off newsletter.

Don’t forget to register for Hacktoberfest by October 31! If you register and have 4 pull requests accepted among the first 40000 participants, you can get a prize. Read more here.

By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.