A complete guide
to ranking and recommendations metrics

How do you judge the quality of ranking and recommender systems? 

Ranking and recommendation systems often focus on the relevance and order of items rather than just the correctness of prediction, as it is in classification or regression. In this guide, we look into the key metrics and explain them step by step.

This guide is for data scientists, ML engineers, product managers, and anyone who deals with operating recommender systems in production. 

‍What you will find in this guide:

• How to evaluate ranking and recommendations. We cover the shared evaluation principles, explaining the data structure, choice of the top-K parameter, and different approaches.
• Deep dives into specific metrics. We explain select metrics in-depth, from ranking metrics like NDCG or MAP to behavioral metrics like serendipity, novelty, and diversity.
• Lots of visuals. All explanations have a lot of illustrations to simplify understanding, even if you are new to the topic.‍
• Modular design. Each article is self-contained, so you can pick and explore the specific metrics and topics of interest. There is no need to read everything cover to cover: start at any point.