Evidently AI Blog - ML Monitoring

Shift happens: we compared 5 methods to detect drift in ML embeddings

May 17, 2023

Monitoring embedding drift is relevant for the production use of LLM and NLP models. We ran experiments to compare 5 drift detection methods. Here is what we found.

ML Monitoring

ML model maintenance. “Should I throw away the drifting features”?

October 26, 2022

Imagine you have a machine learning model in production, and some features are very volatile. Their distributions are not stable. What should you do with those? Should you just throw them away?

ML Monitoring

Pragmatic ML monitoring for your first model. How to prioritize metrics?

August 29, 2022

There is an overwhelming set of potential metrics to monitor. In this blog, we'll try to introduce a reasonable hierarchy.

ML Monitoring

Monitoring ML systems in production. Which metrics should you track?

July 18, 2022

When one mentions "ML monitoring," this can mean many things. Are you tracking service latency? Model accuracy? Data quality? This blog organizes everything one can look at in a single framework.

ML Monitoring

Which test is the best? We compared 5 methods to detect data drift on large datasets

June 20, 2022

We ran an experiment to help build an intuition on how popular drift detection methods behave. In this blog, we share the key takeaways and the code to run the tests on your data.

ML Monitoring

Q&A: ML drift that matters. "How to interpret data and prediction drift together?"

April 25, 2022

Data and prediction drift often need contextual interpretation. In this blog, we walk you through possible scenarios for when you detect these types of drift together or independently.

ML Monitoring

Q&A: Do I need to monitor data drift if I can measure the ML model quality?

December 9, 2021

Even if you can calculate the model quality metric, monitoring data and prediction drift can be often useful. Let’s consider a few examples when it makes sense to track the distributions of the model inputs and outputs.

ML Monitoring

Q&A: What is the difference between outlier detection and data drift detection?

November 4, 2021

When monitoring ML models in production, we can apply different techniques. Data drift and outlier detection are among those. What is the difference? Here is a visual explanation.

ML Monitoring

Can you build a machine learning model to monitor another model?

April 23, 2021

Can you train a machine learning model to predict your model’s mistakes? Nothing stops you from trying. But chances are, you are better off without it.

ML Monitoring

Machine Learning Monitoring, Part 5: Why You Should Care About Data and Concept Drift

November 12, 2020

No model lasts forever. While the data quality can be fine, the model itself can start degrading. A few terms are used in this context. Let’s dive in.

ML Monitoring

Machine Learning Monitoring, Part 4: How To Track Data Quality and Data Integrity

October 23, 2020

A bunch of things can go wrong with the data that goes into a machine learning model. Our goal is to catch them on time.

ML Monitoring

Machine Learning Monitoring, Part 3: What Can Go Wrong With Your Data?

September 9, 2020

Garbage in is garbage out. Input data is a crucial component of a machine learning system. Whether or not you have immediate feedback, your monitoring starts here.

ML Monitoring

Machine Learning Monitoring, Part 2: Who Should Care, and What We Are Missing

August 28, 2020

Who should care about machine learning monitoring? The short answer: everyone who cares about the model's impact on business.

ML Monitoring

Machine Learning Monitoring, Part 1: What It Is and How It Differs

August 24, 2020

Congratulations! Your machine learning model is now live. Many models never make it that far. Some claim, as much as 87% are never deployed.

ML Monitoring

ML in production newsletter

Get a roundup with the best blogs, links and events every month. No spam.

By signing up you agree to receive emails from us. You can opt out any time.

By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.

Deny Accept