VNRE: Flexible and Efficient Acceleration for Network Redundancy Elimination

Xiongzi Ge, Yi Liu, Chengtao Lu, Jim Diehl, David H Du, Liang Zhang, Jian Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Network Redundancy Elimination (NRE) aims to improve network performance by identifying and removing repeated transmission of duplicate content from remote servers. Using a Content-Defined Chunking (CDC) policy, an inline NRE process can obtain a higher Redundancy Elimination (RE) ratio but may suffer from a considerably higher computational requirement than fixed-size chunking. Additionally, the existing work on NRE is either based on IP packet level redundancy elimination or rigidly adopting a CDC policy with a static empirically-decided expected chunk size. These approaches make it difficult for conventional NRE MiddleBoxes to achieve both high network throughput to match the increasing line speeds and a high RE ratio at the same time. In this paper we present a design and implementation of an inline NRE appliance which incorporates an improved FPGA-based scheme to speed up CDC processing to match the ever increasing network line speeds while simultaneously obtaining a high RE ratio. The overhead of Rabin fingerprinting, which is a key component of CDC, is greatly reduced through the use of a record table and registers in the FPGA. To efficiently utilize the hardware resources, the whole NRE process is handled by a Virtualized NRE (VNRE) controller. The uniqueness of this VNRE that we developed lies in its ability to exploit the redundancy patterns of different TCP flows and customize the chunking process to achieve a higher RE ratio. VNRE will first decide if the chunking policy should be either fixed-size chunking or CDC. Then VNRE decides the expected chunk size for the corresponding chunking policy based on the TCP flow patterns. Implemented in a partially reconfigurable FPGA card, our trace driven evaluation demonstrates that the chunking throughput for CDC in one FPGA processing unit outperforms chunking running in a virtual CPU by nearly 3X. Moreover, through the differentiation of chunking policies for each flow, the overall throughput of the VNRE appliance outperforms one with static NRE configurations by 6X to 57X while still guaranteeing a high RE ratio.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages83-92
Number of pages10
ISBN (Electronic)9781509021406
DOIs
StatePublished - Jul 18 2016
Event30th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2016 - Chicago, United States
Duration: May 23 2016May 27 2016

Publication series

NameProceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016

Other

Other30th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2016
Country/TerritoryUnited States
CityChicago
Period5/23/165/27/16

Keywords

  • Content-Defined Chunking
  • FPGA
  • Network Redundancy Elimination
  • RE ratio
  • Throughput

Fingerprint

Dive into the research topics of 'VNRE: Flexible and Efficient Acceleration for Network Redundancy Elimination'. Together they form a unique fingerprint.

Cite this