A memory management system optimized for BDMPI's memory and execution model

Jeremy Iverson, George Karypis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

There is a growing need to perform large computations on small systems, as access to large systems is not widely avail- Able and cannot keep up with the scaling of data. BDMPI was recently introduced as a way of achieving this for applications written in MPI. BDMPI allows the efficient execution of standard MPI programs on systems whose aggregate amount of memory is smaller than that required by the computations and significantly outperforms other approaches. In this paper we present a virtual memory subsystem which we implemented as part of the BDMPI runtime. Our new virtual memory subsystem, which we call SBMA, bypasses the operating system virtual memory manager to take advantage of BDMPI's node-level cooperative multi- Taking. Benchmarking using a synthetic application shows that for the use cases relevant to BDMPI, the overhead incurred by the BDMPI-SBMA system is amortized such that it performs as fast as explicit data movement by the application developer. Furthermore, we tested SBMA with three different classes of applications and our results show that with no modification to the original MPI program, speedups from 2 × 12 over a standard BDMPI implementation can be achieved for the included applications.

Original languageEnglish (US)
Title of host publicationProceedings of the 22nd European MPI Users' Group Meeting, EuroMPI 2015
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450337953
DOIs
StatePublished - Sep 21 2015
Event22nd European MPI Users' Group Meeting, EuroMPI 2015 - Bordeaux, France
Duration: Sep 21 2015Sep 23 2015

Publication series

NameACM International Conference Proceeding Series
Volume21-23-September-2015

Other

Other22nd European MPI Users' Group Meeting, EuroMPI 2015
CountryFrance
CityBordeaux
Period9/21/159/23/15

Keywords

  • Big data
  • Distributed computing
  • MPI
  • Out-of- core
  • Virtual memory

Fingerprint Dive into the research topics of 'A memory management system optimized for BDMPI's memory and execution model'. Together they form a unique fingerprint.

Cite this