Revisiting RoCE on 100GigE

2019-10-01 Tue

net pub

We published an unclassified unlimited release (UUR) technical report.

Abstract

Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) has the potential to provide performance that rivals traditional high performance fabrics. If this potential proves out, significant impacts on system procurement decisions could follow. This work provides a series of small scale performance results which are used to compare and contrast the performance of RoCE-enabled Ethernet with TCP-based Ethernet and an HPC network. Additionally, a discussion of the maturity of RoCE firmware/software stacks and documentation is provided along with useful approaches for probing performance. A detailed description of two experimental setups known to have good RoCE performance is given, including step-by-step configuration and the exact hardware and software revisions employed. At small scales, RoCE is found to have significant performance advantages over "out-of-the-box" TCP protocols and is competitive with state-of-the-art high performance networks. Further examination of RoCE using a wider array of benchmarks and at greater scale is warranted.

Publication

SAND Report Joseph Kenny and Craig Ulmer "RoCE Promising Technology for Ethernet as a High Performance Networking Fabric". SAND2019-13444, October 2019.

100GigE Packet Capture

2019-09-02 Mon

net interns pub

We published an unclassified unlimited release (UUR) technical report.

Publication

SAND Report Haoda Wang, Gavin Baker, Joseph Kenny, and Craig Ulmer "An Initial Investigation of the Design Challenges Associated with Reliable 100GigE Packet Capture". SAND2019-10319, September 2019.