MLOps tutorials

Are you looking for an open-source tool to build an ML monitoring dashboard from scratch? Or, have you already started using the Evidently Reports for ML monitoring but now look at how to turn them into a web app?

One option is to use Evidently together with Streamlit, an open-source Python tool for creating shareable web apps. In this tutorial, you will go through an example of creating and customizing an ML monitoring dashboard using the two open-source libraries.

In this tutorial, we will explore issues affecting the performance of NLP models in production.

We will work with a drug review dataset and go through the following steps:
- Train a simple review classification model, and evaluate its quality on a validation dataset;
- Imitate data quality issues and test their impact on the model accuracy;
- Explore how to detect and debug model quality decay.  

How big of drift is DRIFT? Should I care if only 10% of my features drifted? Should I look at drift week-by-week or month-by-month?

You can look at historical drift in data to understand how your data changes and choose the monitoring thresholds. Here is an example with Evidently, Plotly, Mlflow, and some Python code.

How to decide if any of the trained models is good enough for production use? How to evaluate and compare your models beyond the standard performance checks?

In this tutorial, we will walk through an example of how to assess your model in more detail.

What can go wrong with ML model in production, and how to keep track? Let's walk through an example.

It is a story of how we trained a model, simulated production use, and analyzed its gradual decay.