Parallel tree-projection-based sequence mining algorithms

Valerie Guralnik, George Karypis

Research output: Contribution to journalArticlepeer-review

48 Scopus citations

Abstract

Discovery of sequential patterns is becoming increasingly useful and essential in many scientific and commercial domains. Enormous sizes of available datasets and possibly large number of mined patterns demand efficient, scalable, and parallel algorithms. Even though a number of algorithms have been developed to efficiently parallelize frequent pattern discovery algorithms that are based on the candidate-generation-and-counting framework, the problem of parallelizing the more efficient projection-based algorithms has received relatively little attention and existing parallel formulations have been targeted only toward shared-memory architectures. The irregular and unstructured nature of the task-graph generated by these algorithms and the fact that these tasks operate on overlapping sub-databases makes it challenging to efficiently parallelize these algorithms on scalable distributed-memory parallel computing architectures. In this paper we present and study a variety of distributed-memory parallel algorithms for a tree-projection-based frequent sequence discovery algorithm that are able to minimize the various overheads associated with load imbalance, database overlap, and interprocessor communication. Our experimental evaluation on a 32 processor IBM SP show that these algorithms are capable of achieving good speedups, substantially reducing the amount of the required work to find sequential patterns in large databases.

Original languageEnglish (US)
Pages (from-to)443-472
Number of pages30
JournalParallel Computing
Volume30
Issue number4
DOIs
StatePublished - Apr 1 2004

Keywords

  • Data mining
  • Database projection algorithms
  • Frequent sequential patterns

Fingerprint Dive into the research topics of 'Parallel tree-projection-based sequence mining algorithms'. Together they form a unique fingerprint.

Cite this