Abstract
In this paper, we propose two new parallel formulations of the Apriori algorithm that is used for computing association rules. These new formulations, IDD and HD, address the shortcomings of two previously proposed parallel formulations CD and DD. Unlike the CD algorithm, the IDD algorithm partitions the candidate set intelligently among processors to efficiently parallelize the step of building the hash tree. The IDD algorithm also eliminates the redundant work inherent in DD, and requires substantially smaller communication overhead than DD. But IDD suffers from the added cost due to communication of transactions among processors. HD is a hybrid algorithm that combines the advantages of CD and DD. Experimental results on a 128-processor Cray T3E show that HD scales just as well as the CD algorithm with respect to the number of transactions, and scales as well as IDD with respect to increasing candidate set size.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 337-352 |
| Number of pages | 16 |
| Journal | IEEE Transactions on Knowledge and Data Engineering |
| Volume | 12 |
| Issue number | 3 |
| DOIs | |
| State | Published - 2000 |
Bibliographical note
Funding Information:This work was supported by National Science Foundation (NSF) grant ASC-9634719, Army Research Office contract DA/DAAH04-95-1-0538, the Army High-Performance Computing Research Center under auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAH04-95-2-0003/contract number DAAH04-95-C-008, Cray Research Inc. Fellowship, and IBM partnership award, the content of which does not necessarily reflect the policy of the government and no official endorsement should be inferred. Access to computing facilities was provided by AHPCRC, Minnesota Supercomputer Institute, Cray Research Inc., and NSF grant CDA-9414015.