Using data accessibility for resource selection in large-scale distributed systems

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Large-scale distributed systems provide an attractive scalable infrastructure for network applications. However, the loosely coupled nature of this environment can make data access unpredictable, and in the limit, unavailable. We introduce the notion of accessibility to capture both availability and performance. An increasing number of data-intensive applications require not only considerations of node computation power but also accessibility for adequate job allocations. For instance, selecting a node with intolerably slow connections can offset any benefit to running on a fast node. In this paper, we present accessibility-aware resource selection techniques by which it is possible to choose nodes that will have efficient data access to remote data sources. We show that the local data access observations collected from a node's neighbors are sufficient to characterize accessibility for that node. By conducting trace-based, synthetic experiments on PlanetLab, we show that the resource selection heuristics guided by this principle significantly outperform conventional techniques such as latency-based or random allocations. The suggested techniques are also shown to be stable even under churn despite the loss of prior observations.

Original languageEnglish (US)
Pages (from-to)788-801
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Volume20
Issue number6
DOIs
StatePublished - 2009

Bibliographical note

Funding Information:
This work was supported in part by US National Science Foundation grants ITR-0325949 and CNS-0643505.

Keywords

  • Data Accessibility
  • Data-intensive computing
  • Large-scale distributed systems
  • Passive network performance estimation
  • Resource selection

Fingerprint Dive into the research topics of 'Using data accessibility for resource selection in large-scale distributed systems'. Together they form a unique fingerprint.

Cite this