A highly parallel GPU-based hash accelerator for a data deduplication system

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Scopus citations

Abstract

Recently, data storage systems with data deduplication have been introduced as a method of reducing storage space by eliminating redundant data. In a deduplication storage system, the collision-resistant fingerprint of each data segment must be calculated using a hash algorithm. This paper presents a GPU based accelerator, called g-Dedu, for processing the hash computation of the deduplication system. The g-Dedu accelerator algorithm is especially designed for handling the variable and small size of the data used in a deduplication system, which cannot be processed efficiently by a GPU in a straightforward way. Our data organization approach uses a hierarchical data structure to organize the processing data. A scheduler manages these data for optimal GPU processing. Our patterned data segment approach overcomes some noticeable performance drops resulting from the GPU memory model. Furthermore, different from some previous GPU hash accelerator work, our approach strictly follows the hash processing standard. Using this new approach, g-Dedu achieves 6 times speedup on the SHA-1 computation, and 7.4 times speedup on the SHA-2 computation when compared with a CPU-based mplementation.

Original languageEnglish (US)
Title of host publicationProceedings of the 21st IASTED International Conference on Parallel and Distributed Computing and Systems, PDCS 2009
Pages268-275
Number of pages8
StatePublished - 2009
Event21st IASTED International Conference on Parallel and Distributed Computing and Systems, PDCS 2009 - Cambridge, MA, United States
Duration: Nov 2 2009Nov 4 2009

Publication series

NameProceedings of the IASTED International Conference on Parallel and Distributed Computing and Systems
ISSN (Print)1027-2658

Other

Other21st IASTED International Conference on Parallel and Distributed Computing and Systems, PDCS 2009
Country/TerritoryUnited States
CityCambridge, MA
Period11/2/0911/4/09

Keywords

  • CUDA
  • Deduplication system
  • GPU computing
  • Hash computing

Fingerprint

Dive into the research topics of 'A highly parallel GPU-based hash accelerator for a data deduplication system'. Together they form a unique fingerprint.

Cite this