Property of average precision and its generalization: An examination of evaluation indicator for information retrieval experiments

研究成果: Review article査読

23 被引用数 (Scopus)

抄録

In information retrieval experiments, indicators for measuring effectiveness of the systems or methods are important. Average precision is often used as an indicator for evaluating ranked output of documents in standard retrieval experiments. This report examines some properties of this indicator. First, we clarify mathematically that relevant documents at a higher position in the ranked list contribute much more to increasing the score of average precision. Second, influence of detecting unknown relevant documents on the score is discussed. Third, we examine statistical variation of average precision scores caused by fluctuations of results from relevance judgments. Another issue of this report is to explore evaluation indicators using data of multi-grade relevance. After reviewing indicators proposed by other researchers, i.e., modified sliding ratio, normalized discounted cumulative gain (nDCG), Q-measure and so on, a new indicator, generalized average precision is developed. We compare these indicators empirically using a simple and artificial example.

本文言語English
ページ(範囲)1-19
ページ数19
ジャーナルNII Technical Reports
2005
14
出版ステータスPublished - 2005 9 29
外部発表はい

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications

フィンガープリント 「Property of average precision and its generalization: An examination of evaluation indicator for information retrieval experiments」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル