Large-scale distributed systems provide an attractive scalable infrastructure for network applications. However, the loosely coupled nature of this environment can make data access unpredictable, and in the limit, unavailable. We introduce the notion of accessibility to capture both availability and performance. An increasing number of data-intensive applications require not only considerations of node computation power but also accessibility for adequate job allocations. For instance, selecting a node with intolerably slow connections can offset any benefit to running on a fast node. In this paper, we present accessibility-aware resource selection techniques by which it is possible to choose nodes that will have efficient data access to remote data sources. We show that the local data access observations collected from a node's neighbors are sufficient to characterize accessibility for that node. By conducting trace-based, synthetic experiments on PlanetLab, we show that the resource selection heuristics guided by this principle significantly outperform conventional techniques such as latency-based or random allocations. The suggested techniques are also shown to be stable even under churn despite the loss of prior observations.
|Original language||English (US)|
|Number of pages||14|
|Journal||IEEE Transactions on Parallel and Distributed Systems|
|State||Published - 2009|
Bibliographical noteFunding Information:
This work was supported in part by US National Science Foundation grants ITR-0325949 and CNS-0643505.
- Data Accessibility
- Data-intensive computing
- Large-scale distributed systems
- Passive network performance estimation
- Resource selection