LLM evals + Hacktoberfest = ❤️ Learn how to contribute new LLM evaluation metrics to the open-source Evidently library

A complete guide
to ranking and recommendations metrics

For data scientists, ML engineers, product managers, and all practitioners alike.

How do you judge the quality of ranking and recommender systems?

Ranking and recommendation systems often focus on the relevance and order of items rather than just the correctness of prediction, as it is in classification or regression. In this guide, we look into the key metrics and explain them step by step.

This guide is for data scientists, ML engineers, product managers, and anyone who deals with operating recommender systems in production.

What you will find in this guide:

  • How to evaluate ranking and recommendations. We cover the shared evaluation principles, explaining the data structure, choice of the top-K parameter, and different approaches.
  • Deep dives into specific metrics. We explain select metrics in-depth, from ranking metrics like NDCG or MAP to behavioral metrics like serendipity, novelty, and diversity.
  • Lots of visuals. All explanations have a lot of illustrations to simplify understanding, even if you are new to the topic.
  • Modular design. Each article is self-contained, so you can pick and explore the specific metrics and topics of interest. There is no need to read everything cover to cover: start at any point.

Explore topics

By clicking “Accept”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.