Main memory-based algorithms for efficient parallel aggregation for temporal databases

Dengfeng Gao, Jose Alvin G. Gendrano, Bongki Moon, Richard T. Snodgrass, Minseok Park, Bruce C. Huang, Jim M. Rodrigue

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

The ability to model the temporal dimension is essential to many applications. Furthermore, the rate of increase in database size and stringency of response time requirements has out-paced advancements in processor and mass storage technology, leading to the need for parallel temporal database management systems. In this paper, we introduce a variety of parallel temporal aggregation algorithms for the shared-nothing architecture; these algorithms are based on the sequential aggregation tree algorithm. We are particularly interested in developing parallel algorithms that can maximally exploit available memory to quickly compute large-scale temporal aggregates without intermediate disk writes and reads. Via an empirical study, we found that the number of processing nodes, the partitioning of the data, the placement of results, and the degree of data reduction effected by the aggregation impacted the performance of the algorithms. For distributed result placement, we discovered that greedy time division merge was the obvious choice. For centralized results and high data reduction, pair-wise merge was preferred for a large number of processing nodes; for low data reduction, it only performed well up to 32 nodes. This led us to a centralized variant of greedy time division merge which was best for the remaining cases. We present a cost model that closely predicts the running time of greedy time division merge.

Original languageEnglish (US)
Pages (from-to)123-163
Number of pages41
JournalDistributed and Parallel Databases
Volume16
Issue number2
DOIs
StatePublished - Sep 1 2004

Keywords

  • Aggregation tree algorithm
  • Cost model
  • Data reduction
  • Parallel temporal aggregation
  • Result placement
  • Temporal declustering

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture
  • Information Systems and Management

Fingerprint Dive into the research topics of 'Main memory-based algorithms for efficient parallel aggregation for temporal databases'. Together they form a unique fingerprint.

Cite this