ScalParC: A new scalable and efficient parallel classification algorithm for mining large datasets

Research output: Chapter in Book/Report/Conference proceedingConference contribution

99 Scopus citations

Abstract

In this paper, we present ScalParC (Scalable Parallel Classifier), a new parallel formulation of a decision tree based classification process. Like other state-of-the-art decision tree classifiers such as SPRINT, ScalParC is suited for handling large datasets. We show that existing parallel formulation of SPRINT is unscalable, whereas ScalParC is shown to be scalable in both runtime and memory requirements. We present the experimental results of classifying up to 6.4 million records on up to 128 processors of Cray T3D, in order to demonstrate the scalable behavior of ScalParC. A key component of ScalParC is the parallel hash table. The proposed parallel hashing paradigm can be used to parallelize other algorithms that require many concurrent updates to a large hash table.

Original languageEnglish (US)
Title of host publicationProceedings of the 1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages573-579
Number of pages7
Volume1998-March
ISBN (Electronic)0818684038, 9780818684036
DOIs
StatePublished - Jan 1 1998
Event1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998 - Orlando, United States
Duration: Mar 30 1998Apr 3 1998

Publication series

NameProceedings of the International Parallel Processing Symposium, IPPS
ISSN (Print)1063-7133

Other

Other1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998
CountryUnited States
CityOrlando
Period3/30/984/3/98

Fingerprint

Dive into the research topics of 'ScalParC: A new scalable and efficient parallel classification algorithm for mining large datasets'. Together they form a unique fingerprint.

Cite this