Coordinating parallel hierarchical storage management in object-base cluster file systems

Dingshan He, Xianbo Zhang, David H Du, Gary Grider

Research output: Contribution to journalConference articlepeer-review

10 Scopus citations

Abstract

Object-based storage technology enables building large-scale and highly-scalable cluster file systems using commodity hardware and software components. On the other hand, a hierarchy of storage subsystems with different costs and performance should be incorporated into such systems to make them affordable and cost-effective. Existing SAN-based (block-based) cluster solutions suffer from slow data movement between storage levels due to their single-archiving-point architecture. In this paper, we propose a novel parallel data moving architecture in object-based cluster file systems. Data movements are coordinated and performed in parallel between multiple pairs of storage subsystems. In addition, data movements are fully automated and transparent to users. Our proposed parallel data moving architecture is prototyped on the Lustre file system. Performance study shows that our scheme can scale up easily by adding more pairs of hierarchical storage devices.

Fingerprint

Dive into the research topics of 'Coordinating parallel hierarchical storage management in object-base cluster file systems'. Together they form a unique fingerprint.

Cite this