Give us a star ⭐️ on GitHub to support the project!
🚀 Join us November 30 for the Evidently monthly demo and Q&A. Register now →

Open-source ML observability course 📈

We are launching a free Open-source ML observability course for data scientists and ML engineers!

Join us to learn production ML monitoring.

🗓 Start date: October 16, 2023.

Evidently Hacktoberfest 2022

What you will learn

The foundations of ML observability: from exploratory data analysis and model evaluation pre-deployment to continuous production monitoring and debugging.

  • How to evaluate ML model before you deploy and design your ML monitoring strategy.
  • How to test data quality in production ML pipelines.
  • How to detect and interpret data and prediction drift.
  • How to continuously monitor ML models for batch and real-time systems.
  • How to monitor changes in text data, and quality of NLP and LLM systems.
  • How to debug model performance decay.
  • Different ML monitoring architectures.
Open-sourse ML observability course
About the course:
✅ Free
🎥 6 modules with on-demand videos
💻 Practical code examples
⭐️ Certificate upon completion

Sign up to join us!

The Open-source ML observability course starts on October 16, 2023.
Sign up to save your seat and receive weekly course updates.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Welcome to the course

Course materials

Jump to the course website for all useful links and course notes. We will continue to publish new materials throughout the course.

Open-source ML observability course from Evidently AI

Course syllabus

Our course is organized into six modules. You can follow the complete course syllabus or pick only the modules that are most relevant to you.

📚 Module 1: Introduction to ML monitoring and observability
Basics of ML monitoring and observability.

📈 Module 2: ML monitoring metrics: model quality, data quality, data drift
Metrics and evaluation methods for structured data.

🔡 Module 3: ML monitoring for unstructured data: NLP, LLM and embeddings
Metrics and evaluation methods for unstructured data.

🏗 Module 4: Designing effective ML monitoring
Key questions to consider when customizing ML monitoring for your model.

✅ Module 5: ML pipelines validation and testing
How to deploy an end-to-end pipeline to check data and ML model quality.

📊 Module 6: Deploying an ML monitoring dashboard
How to deploy an ML monitoring service and design a live dashboard.

SIGN UP for free
Emeli Dral

Emeli Dral

Co-founder and CTO
Evidently AI
Emeli Dral LinkedIn profile

Meet the instructor

Emeli Dral is a Co-founder and CTO at Evidently AI, a startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models.

Earlier, she co-founded an industrial AI startup and served as the Chief Data Scientist at Yandex Data Factory. She led over 50 applied ML projects for various industries: from banking to manufacturing. Emeli is a data science lecturer at GSOM SpBU and Harbour.Space University. She is a co-author of the Machine Learning and Data Analysis curriculum at Coursera with over 150,000 students.


Is the course free?

Yes, the course is 100% free.

What is the course format?

The course is organized into six modules. Each module consists of on-demand videos and practical code examples. We’ll release each new module every week, starting October 16, 2023. 

To earn a certificate of completion, you must enroll on the course platform and complete the quizzes and a final assignment before December 1, 2023.  

This is optional — you can also learn at your own pace and go through the videos, which will remain publicly available after the course cohort. You can follow the complete course syllabus or choose only the most relevant modules.

How long does it take to complete the course?

The course lasts seven weeks. There are six modules with learning materials, and one extra week to complete the final assignment.

However, you can also learn at your own pace — whatever works best for you!

Will I get a course certificate?

You will receive a certificate if you join the course and successfully complete all the assignments before December 1, 2023.

Note that the option to receive the certificate is available only to those who participate in the course cohort. If you are taking the course after December 1, 2023, you can freely access the materials, but you won’t be eligible to receive the certificate.

Are there any course prerequisites?

There are both theoretical and code-focused modules that require knowledge of Python. But no worries: we will walk you through the code! You can also skip these parts and still learn a lot.

What if I need help?

Have a question or just want to say “Hi”? Jump to our Discord #-ml-observability-course channel to chat with fellow learners and get support from the course team. 

We will also host Q&A sessions with the course instructor, Emeli Dral, on a Zoom call once per week.

Where can I find course materials?

We’ll send all the information to those who signed up for the course before October 16, 2023. Please make sure that you sign up for the course to stay updated.

Jump to the course website for all useful links and course notes. We will continue to publish new materials throughout the course.

Is there a newsletter?

Yes, sign up here.

Will course materials still be available after the course?

Yes! All course materials are public so you can get back to them at your convenience, during or after the course. 

How to participate?

In the Evidently GitHub repository, we added a special set of issues labeled “hacktoberfest."

step 1
Check out the guide

Head to Evidently Hacktoberfest guide for clear steps and detailed examples.

step 2
Pick an issue

Check out the issues we prepared for Hacktoberfest. You can pick one of them or propose a different metric.

step 3
Submit pull request

Choose the drift method you want to implement and submit your pull request!

step 4
Get feedback

Wait for your pull request to be reviewed.

What is Evidently?

Evidently is an open-source Python library for data scientists and ML engineers. It helps evaluate, test, and monitor the performance of ML models from validation to production. You can check it out on GitHub or explore the documentation.

What is Hacktoberfest?

Hacktoberfest is an annual event to celebrate open-source and encourage contributions. It runs for the 9th time this year. Eligible participants will get prizes from DigitalOcean. But first and foremost, it is a great reason to create your first (or hundredth) pull request! You can check out the complete rules here.

How can I contribute?

Evidently is an open-source project, and is always open for contributions. For Hacktoberfest, we added a special set of issues labeled “hacktoberfest” to the Evidently GitHub repository. We invite data scientists to dip their toes into open-source contribution and help us add new statistical metrics and tests to detect data drift for production ML models. Check out Hacktoberfest issues on our GitHub. Head to the Evidently Hacktoberfest guide for clear steps and detailed examples. Sign up to receive the kick-off newsletter.

Don’t forget to register for Hacktoberfest by October 31! If you register and have 4 pull requests accepted among the first 40000 participants, you can get a prize. Read more here.

By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.