Jim Plank's Software
Information Theory Explorations
Chao Tian, James S. Plank and Brent Hurst,
``An Open-Source Toolbox for Computer-Aided Investigation on the Fundamental Limits of Information Systems, Version 0.1,''
https://github.com/ct2641/CAI/releases/tag/0.1, October, 2019.
Also available as a paper at https://arxiv.org/abs/1910.08567.
Erasure Coding and Galois Field Arithmetic:
GF-Complete and Jerasure version 2.0 have been removed from web sites
at the University of Tennessee and bitbucket.org. The source code
of both projects have
been archived for non-commercial use at
http://web.eecs.utk.edu/~jplank/plank/papers/Jerasure-2-0-Archive.pdf and
http://web.eecs.utk.edu/~jplank/plank/papers/GF-Complete-Archive.pdf
On this page I (James Plank) am providing notice that:
- GF-Complete and Jerasure versions 2.0 and later are no longer supported.
- StreamScale, Inc. offers a similar solution for commercial purposes.
- I offer no representations or warranties in general about StreamScale's products.
- I have verified that StreamScale's solution is faster than GF-Complete or Jerasure in at least some respects.
- StreamScale, Inc. asserts that the use of GF-Complete (particularly as part of
Jerasure 2.0 or later) or any similar software, method or code for erasure coding
infringes StreamScale's issued United States Patent No. 8,683,296.
- I express no opinion on StreamScale's claims, but I believe that parties should be
aware that StreamScale asserts such claims.
If you wish to cite GF-Complete or Jerasure in a paper, please cite their technical report
versions:
Over the years, I have exported multiple Galois Field arithmetic implementations, and multiple
revisions of Jerasure. Here are their descriptions:
- GF-Complete: See above.
- ``Fast Galois Field Arithmetic Library in C/C++.''
April, 2007. This implements multiplication/division in
GF(2w)), for any w from 1 to 32. It uses tables for smaller w,
log tables for larger w, "split" tables for w=32, and some pretty slow implementations
for larger values of w that are not 32. This is what Jerasure used until revision 2.0,
when it switched to GF-Complete.
- GFLIB. June, 2003.
This is an older library that contains Galois Arithmetic code for w=8 and w=16
only. The performance of w=8 is lacking as compared to the 2007 code. It does,
however, implement Vandermonde-matrix based Reed-Solomon coding and decoding.
- Jerasure version 2.0. January, 2014. See above.
- Jerasure version 1.2A: 2011. This is pretty much the same as Jerasure 1.2, only released under the New BSD License rather
than the LGPL License. (The link is a tar file).
- Jerasure version 1.2:
August, 2008. Added an encoder/decoder, better examples, Blaum-Roth codes and the Liber8tion
code to Jerasure.
- Jerasure version 1.0:
Reed-Solomon coding, Cauchy Reed-Solomon coding, Liberation coding, general matrix and bit-matrix
coding for horizontal, systematic erasure codes.
Other software for Erasure Coding
Open Source Encoder and Decoder for SD Erasure Codes - Revision 2.0 -- these are from my 2014 TOS paper with Mario Blaum.
The code here depended on GF-Complete, which has been removed from my web sites.
To make use of these, you will need to port them to a different Galois Field arithmetic library.
Open Source Encoder and Decoder for SD Erasure Codes -- these are from my 2013 FAST paper with Mario Blaum and Jim Hafner.
The code here depended on GF-Complete, which has been removed from my web sites.
To make use of these, you will need to port them to a different Galois Field arithmetic library.
Uber-CSHR
and X-Sets: C++ Programs for Optimizing Matrix-Based Erasure Codes for Fault-Tolerant Storage Systems, December, 2010. These are the programs for my 2012 DSN paper.
Tutorial Material for Erasure Coding
To make best use of these codes, please read the following tutorial material:
- James S. Plank,
``Erasure Codes for
Storage Systems: A Brief Primer,''
;login: the Usenix Magazine, December 2013, Volume 38, Number 6. Usenix Association.
- James S. Plank and Cheng Huang: Slides for our 2013 FAST Tutorial on Erasure Coding for Storage
- James S. Plank Slides for my 2005 FAST Tutorial on Erasure Coding for Storage.
- James S. Plank, ``A
Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems,''
Software -- Practice & Experience, 27(9), September, 1997, pp. 995-1012.
- James S. Plank and Ying Ding,
``Note:
Correction to the 1997 Tutorial on Reed-Solomon Coding,''
Software -- Practice & Experience, 35(2), February, 2005, pp. 189-194.
All the LoCI software may be found at the LoCI web site.
Jgraph is a program that takes the description of a graph or graphs as
input, and produces a postscript file on the standard output. Jgraph
is ideal for plotting any mixture of scatter point graphs, line
graphs, and/or bar graphs, and embedding the output into LaTeX, or
any other text processing system that can read postscript.
Ickp is a library that enables users of the Intel iPSC/860 to checkpoint
the execution state of their programs to disk. The major use for ickp
is to provide fault-tolerance and/or coarse-grained job swapping to
iPSC/860 users.
Libckpt is a portable checkpointing tool for Unix.
It provides a mechanism for enabling fault-tolerance for
long-running programs.
Libckpt implements most optimizations that have been proposed
to improve the performance of checkpointing.
Additionally, it implements ``user-directed'' checkpointing, a new
optimization unique to Libckpt.
This page contains pointers to various application programs that
I have used for checkpointing.
This is a library that contains the red-black tree, doubly-linked list and
fields code below. It was updated in 1999, and is easier to use than
the previous code that I had. The old code may be obtained from the links
below, but I recommend that you use this code.
This contains code for using generic red-black trees. Red-black
trees are a dynamic data structure in which all operations are
guaranteed to perform in log time. The code is ansi or non-ansi C,
and is very easy to use.
Dlist: Code for Doubly-Linked
Lists
This contains C code for using generic doubly-linked lists.
Fields: Code for fields processing
This makes input processing in C easier, and is used in CS360/CS460.