Deploying machine learning on edge devices is becoming increasingly important, driven by new applications such as smart homes, smart cities, and autonomous vehicles. Unfortunately, it is challenging to deploy deep neural networks (DNNs) on resource-constrained devices. These workloads are computationally intensive and often require cloud-like resources. Prior solutions attempted to address these challenges by either sacrificing accuracy or by relying on cloud resources for assistance. In this paper, we propose a containerized partition-based runtime adaptive convolutional neural network (CNN) acceleration framework for Internet of Things (IoT) environments. The framework leverages spatial partitioning techniques through convolution layer fusion to dynamically select the optimal partition according to the availability of computational resources and network conditions. By containerizing each partition, we simplify the model update and deployment with Docker and Kubernetes to efficiently handle runtime resource management and scheduling of containers.
|Original language||English (US)|
|State||Published - 2019|
|Event||2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2019, co-located with USENIX ATC 2019 - Renton, United States|
Duration: Jul 9 2019 → …
|Conference||2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2019, co-located with USENIX ATC 2019|
|Period||7/9/19 → …|
Bibliographical noteFunding Information:
The authors would like to thank the anonymous reviewers for their feedback. We also extend special thanks to Irfan Ahmad for suggestions on the camera ready. This work was supported in part by NSF Award 1439622, 1812537 and NSF XPS Award 60053525.