Abstract
This paper presents ST-Hadoop; the first full-fledged open-source MapReduce framework with a native support for spatio-temporal data. ST-Hadoop is a comprehensive extension to Hadoop and SpatialHadoop that injects spatio-temporal data awareness inside each of their layers, mainly, language, indexing, and operations layers. In the language layer, ST-Hadoop provides built in spatio-temporal data types and operations. In the indexing layer, ST-Hadoop spatiotemporally loads and divides data across computation nodes in Hadoop Distributed File System in a way that mimics spatio-temporal index structures, which result in achieving orders of magnitude better performance than Hadoop and SpatialHadoop when dealing with spatio-temporal data and queries. In the operations layer, ST-Hadoop shipped with support for two fundamental spatio-temporal queries, namely, spatio-temporal range and join queries. Extensibility of ST-Hadoop allows others to expand features and operations easily using similar approach described in the paper. Extensive experiments conducted on large-scale dataset of size 10Â TB that contains over 1 Billion spatio-temporal records, to show that ST-Hadoop achieves orders of magnitude better performance than Hadoop and SpaitalHadoop when dealing with spatio-temporal data and operations. The key idea behind the performance gained in ST-Hadoop is its ability in indexing spatio-temporal data within Hadoop Distributed File System.
| Original language | English (US) |
|---|---|
| Title of host publication | Advances in Spatial and Temporal Databases - 15th International Symposium, SSTD 2017, Proceedings |
| Editors | Wei-Shinn Ku, Agnes Voisard, Haiquan Chen, Chang-Tien Lu, Siva Ravada, Matthias Renz, Yan Huang, Michael Gertz, Liang Tang, Chengyang Zhang, Erik Hoel, Xiaofang Zhou |
| Publisher | Springer Verlag |
| Pages | 84-104 |
| Number of pages | 21 |
| ISBN (Print) | 9783319643663 |
| DOIs | |
| State | Published - 2017 |
| Event | 15th International Symposium on Spatial and Temporal Databases, SSTD 2017 - Arlington, United States Duration: Aug 21 2017 → Aug 23 2017 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 10411 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Other
| Other | 15th International Symposium on Spatial and Temporal Databases, SSTD 2017 |
|---|---|
| Country/Territory | United States |
| City | Arlington |
| Period | 8/21/17 → 8/23/17 |
Bibliographical note
Funding Information:This work is partially supported by the National Science Foundation, USA, under Grants IIS-1525953, CNS-1512877, IIS-1218168, and by a scholarship from the College of Computers & Information Systems, Umm Al-Qura University, Makkah, Saudi Arabia.
Publisher Copyright:
© Springer International Publishing AG 2017.