A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries For Storage

James S. Plank, EECS Department, University of Tennessee,
Jianqiang Luo, Computer Science Department, Wayne State University,
Catherine D. Schuman, EECS Department, University of Tennessee,
Lihao Xu, Computer Science Department, Wayne State University,
Zooko Wilcox-O'Hearn, AllMyData, Inc.

Appearing in FAST-2009: 7th USENIX Conference on File and Storage Technologies San Francisco, California, February, 2009.

PDF of the paper.

Abstract

Over the past five years, large-scale storage installations have required fault-protection beyond RAID-5, leading to a flurry of research on and development of erasure codes for multiple disk failures. Numerous open-source implementations of various coding techniques are available to the general public. In this paper, we perform a head-to-head comparison of these implementations in encoding and decoding scenarios. Our goals are to compare codes and implementations, to discern whether theory matches practice, and to demonstrate how parameter selection, especially as it concerns memory, has a significant impact on a code's performance. Additional benefits are to give storage system designers an idea of what to expect in terms of coding performance when designing their storage systems, and to identify the places where further erasure coding research can have the most impact.

Citation Information

Plain Text:

.inproceedings  pls:09:os
author          J. S. Plank and J. Luo and C. D. Schuman and L. Xu and Z. Wilcox-O'Hearn
title           A  Performance Evaluation and Examination of Open-Source Erasure Coding
                Libraries For Storage
booktitle       FAST-2009: 7th Usenix Conference on File and Storage Technologies
month           February
year            2009
where           http://web.eecs.utk.edu/~jplank/plank/papers/FAST-2009.html

Bibtex:

@INPROCEEDINGS{pls:09:os,
        author = "J. S. Plank and J. Luo and C. D. Schuman and L. Xu and Z. Wilcox-O'Hearn",
        title = "A  Performance Evaluation and Examination of Open-Source Erasure Coding
                Libraries For Storage",
        booktitle = "FAST-2009: 7th Usenix Conference on File and Storage Technologies",
        month = "February",
        year = "2009",
        where = "http://web.eecs.utk.edu/~jplank/plank/papers/FAST-2009.html"
}