RCP-nDCG@10: A New Approach to Enterprise Retrieval Quality | Cohere

Introduces RCP-nDCG@10, a new evaluation methodology for retrieval systems that addresses the limitations of traditional nDCG by using a calibrated AI judge to assess relevance against an explicit rubric, rather than relying solely on pre-existing, potentially incomplete, relevance labels (qrels). This method was validated against human judgment and showed higher agreement with human preferences. The post details the methodology, its validation, and its adoption as the optimization target for Cohere's next generation of Embed and Rerank models.