Scalable Online Index Construction with Multi-core cpus

Hiroyuki Yamada, Motomichi Toyama

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Inverted index is a core element of current text retrieval systems. They can be dynamically constructed using online indexing approaches in the environment which even a small delay in timeliness cannot be tolerated, and the index must always be queryable and up to date. Recently, efficient online index construction schemes have been proposed, however, previous works have not focused on scalability with the modern commodity hardware resources such as multi-core CPUs. In this paper, we propose a scalable online index construction method that better utilizes multicore CPUs. Using experiments on 30 GB of web data, we demonstrate the efficiency of our method in practice, showing that it dramatically reduces online index construction time without sacrificing query performance.

Original languageEnglish
Title of host publicationConferences in Research and Practice in Information Technology Series
Pages29-36
Number of pages8
Volume104
Publication statusPublished - 2010
Event21st Australasian Database Conference, ADC 2010 - Brisbane, QLD, Australia
Duration: 2010 Jan 182010 Jan 21

Other

Other21st Australasian Database Conference, ADC 2010
CountryAustralia
CityBrisbane, QLD
Period10/1/1810/1/21

Fingerprint

Program processors
Computer hardware
Scalability
Experiments

Keywords

  • Index maintenance
  • Information retrieval
  • Inverted index
  • Online index construction
  • Text databases

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems
  • Software

Cite this

Yamada, H., & Toyama, M. (2010). Scalable Online Index Construction with Multi-core cpus. In Conferences in Research and Practice in Information Technology Series (Vol. 104, pp. 29-36)

Scalable Online Index Construction with Multi-core cpus. / Yamada, Hiroyuki; Toyama, Motomichi.

Conferences in Research and Practice in Information Technology Series. Vol. 104 2010. p. 29-36.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yamada, H & Toyama, M 2010, Scalable Online Index Construction with Multi-core cpus. in Conferences in Research and Practice in Information Technology Series. vol. 104, pp. 29-36, 21st Australasian Database Conference, ADC 2010, Brisbane, QLD, Australia, 10/1/18.
Yamada H, Toyama M. Scalable Online Index Construction with Multi-core cpus. In Conferences in Research and Practice in Information Technology Series. Vol. 104. 2010. p. 29-36
Yamada, Hiroyuki ; Toyama, Motomichi. / Scalable Online Index Construction with Multi-core cpus. Conferences in Research and Practice in Information Technology Series. Vol. 104 2010. pp. 29-36
@inproceedings{e0a3a6a2d9554f59957404595e3eb3e1,
title = "Scalable Online Index Construction with Multi-core cpus",
abstract = "Inverted index is a core element of current text retrieval systems. They can be dynamically constructed using online indexing approaches in the environment which even a small delay in timeliness cannot be tolerated, and the index must always be queryable and up to date. Recently, efficient online index construction schemes have been proposed, however, previous works have not focused on scalability with the modern commodity hardware resources such as multi-core CPUs. In this paper, we propose a scalable online index construction method that better utilizes multicore CPUs. Using experiments on 30 GB of web data, we demonstrate the efficiency of our method in practice, showing that it dramatically reduces online index construction time without sacrificing query performance.",
keywords = "Index maintenance, Information retrieval, Inverted index, Online index construction, Text databases",
author = "Hiroyuki Yamada and Motomichi Toyama",
year = "2010",
language = "English",
volume = "104",
pages = "29--36",
booktitle = "Conferences in Research and Practice in Information Technology Series",

}

TY - GEN

T1 - Scalable Online Index Construction with Multi-core cpus

AU - Yamada, Hiroyuki

AU - Toyama, Motomichi

PY - 2010

Y1 - 2010

N2 - Inverted index is a core element of current text retrieval systems. They can be dynamically constructed using online indexing approaches in the environment which even a small delay in timeliness cannot be tolerated, and the index must always be queryable and up to date. Recently, efficient online index construction schemes have been proposed, however, previous works have not focused on scalability with the modern commodity hardware resources such as multi-core CPUs. In this paper, we propose a scalable online index construction method that better utilizes multicore CPUs. Using experiments on 30 GB of web data, we demonstrate the efficiency of our method in practice, showing that it dramatically reduces online index construction time without sacrificing query performance.

AB - Inverted index is a core element of current text retrieval systems. They can be dynamically constructed using online indexing approaches in the environment which even a small delay in timeliness cannot be tolerated, and the index must always be queryable and up to date. Recently, efficient online index construction schemes have been proposed, however, previous works have not focused on scalability with the modern commodity hardware resources such as multi-core CPUs. In this paper, we propose a scalable online index construction method that better utilizes multicore CPUs. Using experiments on 30 GB of web data, we demonstrate the efficiency of our method in practice, showing that it dramatically reduces online index construction time without sacrificing query performance.

KW - Index maintenance

KW - Information retrieval

KW - Inverted index

KW - Online index construction

KW - Text databases

UR - http://www.scopus.com/inward/record.url?scp=84873416739&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873416739&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84873416739

VL - 104

SP - 29

EP - 36

BT - Conferences in Research and Practice in Information Technology Series

ER -