Term disambiguation techniques based on target document collection for cross-language information retrieval: An empirical comparison of performance between techniques

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Dictionary-based query translation for cross-language information retrieval often yields various translation candidates having different meanings for a source term in the query. This paper examines methods for solving the ambiguity of translations based on only the target document collections. First, we discuss two kinds of disambiguation technique: (1) one is a method using term co-occurrence statistics in the collection, and (2) a technique based on pseudo-relevance feedback. Next, these techniques are empirically compared using the CLEF 2003 test collection for German to Italian bilingual searches, which are executed by using English language as a pivot. The experiments showed that a variation of term co-occurrence based techniques, in which the best sequence algorithm for selecting translations is used with the Cosine coefficient, is dominant, and that the PRF method shows comparable high search performance, although statistical tests did not sufficiently support these conclusions. Furthermore, we repeat the same experiments for the case of French to Italian (pivot) and English to Italian (non-pivot) searches on the same CLEF 2003 test collection in order to verity our findings. Again, similar results were observed except that the Dice coefficient outperforms slightly the Cosine coefficient in the case of disambiguation based on term co-occurrence for English to Italian searches.

Original languageEnglish
Pages (from-to)103-120
Number of pages18
JournalInformation Processing and Management
Volume43
Issue number1
DOIs
Publication statusPublished - 2007 Jan

Fingerprint

Query languages
information retrieval
Statistical tests
Glossaries
language
performance
Experiments
Statistics
Feedback
statistical test
experiment
dictionary
English language
candidacy
statistics
Cross-language information retrieval
Coefficients
Experiment
Test collections

Keywords

  • Cross-language information retrieval
  • Dictionary-based query translation
  • Pseudo-relevance feedback
  • Statistical comparison
  • Term co-occurrence statistics
  • Term disambiguation

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Library and Information Sciences

Cite this

@article{696f875c03b948e5af2e96cb5bb5f2e6,
title = "Term disambiguation techniques based on target document collection for cross-language information retrieval: An empirical comparison of performance between techniques",
abstract = "Dictionary-based query translation for cross-language information retrieval often yields various translation candidates having different meanings for a source term in the query. This paper examines methods for solving the ambiguity of translations based on only the target document collections. First, we discuss two kinds of disambiguation technique: (1) one is a method using term co-occurrence statistics in the collection, and (2) a technique based on pseudo-relevance feedback. Next, these techniques are empirically compared using the CLEF 2003 test collection for German to Italian bilingual searches, which are executed by using English language as a pivot. The experiments showed that a variation of term co-occurrence based techniques, in which the best sequence algorithm for selecting translations is used with the Cosine coefficient, is dominant, and that the PRF method shows comparable high search performance, although statistical tests did not sufficiently support these conclusions. Furthermore, we repeat the same experiments for the case of French to Italian (pivot) and English to Italian (non-pivot) searches on the same CLEF 2003 test collection in order to verity our findings. Again, similar results were observed except that the Dice coefficient outperforms slightly the Cosine coefficient in the case of disambiguation based on term co-occurrence for English to Italian searches.",
keywords = "Cross-language information retrieval, Dictionary-based query translation, Pseudo-relevance feedback, Statistical comparison, Term co-occurrence statistics, Term disambiguation",
author = "Kazuaki Kishida",
year = "2007",
month = "1",
doi = "10.1016/j.ipm.2006.04.006",
language = "English",
volume = "43",
pages = "103--120",
journal = "Information Processing and Management",
issn = "0306-4573",
publisher = "Elsevier Limited",
number = "1",

}

TY - JOUR

T1 - Term disambiguation techniques based on target document collection for cross-language information retrieval

T2 - An empirical comparison of performance between techniques

AU - Kishida, Kazuaki

PY - 2007/1

Y1 - 2007/1

N2 - Dictionary-based query translation for cross-language information retrieval often yields various translation candidates having different meanings for a source term in the query. This paper examines methods for solving the ambiguity of translations based on only the target document collections. First, we discuss two kinds of disambiguation technique: (1) one is a method using term co-occurrence statistics in the collection, and (2) a technique based on pseudo-relevance feedback. Next, these techniques are empirically compared using the CLEF 2003 test collection for German to Italian bilingual searches, which are executed by using English language as a pivot. The experiments showed that a variation of term co-occurrence based techniques, in which the best sequence algorithm for selecting translations is used with the Cosine coefficient, is dominant, and that the PRF method shows comparable high search performance, although statistical tests did not sufficiently support these conclusions. Furthermore, we repeat the same experiments for the case of French to Italian (pivot) and English to Italian (non-pivot) searches on the same CLEF 2003 test collection in order to verity our findings. Again, similar results were observed except that the Dice coefficient outperforms slightly the Cosine coefficient in the case of disambiguation based on term co-occurrence for English to Italian searches.

AB - Dictionary-based query translation for cross-language information retrieval often yields various translation candidates having different meanings for a source term in the query. This paper examines methods for solving the ambiguity of translations based on only the target document collections. First, we discuss two kinds of disambiguation technique: (1) one is a method using term co-occurrence statistics in the collection, and (2) a technique based on pseudo-relevance feedback. Next, these techniques are empirically compared using the CLEF 2003 test collection for German to Italian bilingual searches, which are executed by using English language as a pivot. The experiments showed that a variation of term co-occurrence based techniques, in which the best sequence algorithm for selecting translations is used with the Cosine coefficient, is dominant, and that the PRF method shows comparable high search performance, although statistical tests did not sufficiently support these conclusions. Furthermore, we repeat the same experiments for the case of French to Italian (pivot) and English to Italian (non-pivot) searches on the same CLEF 2003 test collection in order to verity our findings. Again, similar results were observed except that the Dice coefficient outperforms slightly the Cosine coefficient in the case of disambiguation based on term co-occurrence for English to Italian searches.

KW - Cross-language information retrieval

KW - Dictionary-based query translation

KW - Pseudo-relevance feedback

KW - Statistical comparison

KW - Term co-occurrence statistics

KW - Term disambiguation

UR - http://www.scopus.com/inward/record.url?scp=33748295501&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748295501&partnerID=8YFLogxK

U2 - 10.1016/j.ipm.2006.04.006

DO - 10.1016/j.ipm.2006.04.006

M3 - Article

AN - SCOPUS:33748295501

VL - 43

SP - 103

EP - 120

JO - Information Processing and Management

JF - Information Processing and Management

SN - 0306-4573

IS - 1

ER -