Loading...
Please wait, while we are loading the content...
Similar Documents
A Theoretical Analysis of Normalized Discounted Cumulative Gain (NDCG) Ranking Measures
| Content Provider | CiteSeerX |
|---|---|
| Author | Wang, Yining Wang, Liwei Li, Yuanzhi He, Di Chen, Wei Liu, Tie-Yan |
| Abstract | A central problem in ranking is to design a measure for evaluation of ranking functions. In this paper we study, from a theoretical perspective, the Normalized Discounted Cumulative Gain (NDCG) which is a family of ranking measures widely used in practice. Although there are extensive empirical studies of NDCG, little is known about its theoretical prop-erties. We rst show that, whatever the ranking function is, the standard NDCG which adopts a logarithmic discount, converges to 1 as the number of items to rank goes to innity. On the rst sight, this result seems to imply that the standard NDCG cannot dierentiate good and bad ranking functions on large datasets, contradicting to its empirical success in many applications. In order to have a deeper understanding of the general NDCG rank-ing measures, we propose a notion referred to as consistent distinguishability. This notion captures the intuition that a ranking measure should have such a property: For every pair of substantially dierent ranking functions, the ranking measure can decide which one is better in a consistent manner on almost all datasets. We show that the standard NDCG has consistent distinguishability although it converges to the same limit for all ranking |
| File Format | |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
