Scalable parallel data mining for association rules

Research output: Contribution to journalArticlepeer-review

131 Scopus citations

Abstract

In this paper, we propose two new parallel formulations of the Apriori algorithm that is used for computing association rules. These new formulations, IDD and HD, address the shortcomings of two previously proposed parallel formulations CD and DD. Unlike the CD algorithm, the IDD algorithm partitions the candidate set intelligently among processors to efficiently parallelize the step of building the hash tree. The IDD algorithm also eliminates the redundant work inherent in DD, and requires substantially smaller communication overhead than DD. But IDD suffers from the added cost due to communication of transactions among processors. HD is a hybrid algorithm that combines the advantages of CD and DD. Experimental results on a 128-processor Cray T3E show that HD scales just as well as the CD algorithm with respect to the number of transactions, and scales as well as IDD with respect to increasing candidate set size.

Original languageEnglish (US)
Pages (from-to)337-352
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume12
Issue number3
DOIs
StatePublished - 2000

Bibliographical note

Funding Information:
This work was supported by National Science Foundation (NSF) grant ASC-9634719, Army Research Office contract DA/DAAH04-95-1-0538, the Army High-Performance Computing Research Center under auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAH04-95-2-0003/contract number DAAH04-95-C-008, Cray Research Inc. Fellowship, and IBM partnership award, the content of which does not necessarily reflect the policy of the government and no official endorsement should be inferred. Access to computing facilities was provided by AHPCRC, Minnesota Supercomputer Institute, Cray Research Inc., and NSF grant CDA-9414015.

Fingerprint

Dive into the research topics of 'Scalable parallel data mining for association rules'. Together they form a unique fingerprint.

Cite this