A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications

Catherine D. Schuman

James S. Plank

August 8, 2008

Technical Report UT-CS-08-625
Department of Computer Science
University of Tennessee
Knoxville, TN 37996

Abstract

Erasure coding is a fundemental technique to prevent data loss in storage systems composed of multiple disks. Recently, there have been multiple open-source implementations of a variety of erasure codes. In this work, we present a comparison of the performance of various codes and implementations, concentrating on encoding and decoding. It is hard to draw overarching conclusions from a single performance study. However, performance data is important to gain an understanding of the real-life performance ramifications of code properties and implementation decisions. The significance of this paper is to guide those who use and design codes, so that they may be able to predict what performance to expect when using an erasure code. One important, although obvious, conclusion is that reducing cache misses is more important than reducing XOR operations.

Citation Information

Plain Text:

author          C. D. Schuman and J. S. Plank
title           A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications
institution     University of Tennessee
month           August
year            2008
number          UT-CS-08-625
where           http://web.eecs.utk.edu/~jplank/plank/papers/CS-08-625.html

Bibtex:

@TECHREPORT{sp:08:pc,
        author = "C. D. Schuman and J. S. Plank",
        title = "A Performance Comparison of Open-Source Erasure Coding Libraries for Storage Applications",
        institution = "University of Tennessee",
        month = "August",
        year = "2008",
        number = "UT-CS-08-625",
        where = "http://web.eecs.utk.edu/~jplank/plank/papers/CS-08-625.html"
}