gzip vs. pigz vs. bzip2 vs. pbzip2

nospam@example.com (christian) — Fri, 01 Jul 2011 23:22:09 +0000

Shortly after the last benchmark, I came across pigz (parallel gzip) and a bigger (real-world) task to complete:

$ time gzip -c file.tar > file.tar
real     41m52.636s
user     33m58.392s
sys       2m26.903s

$ time pigz -c file.tar > file.tar.pigz
real     18m34.894s
user     54m07.784s
sys       3m47.910s

$ time bzip2 -c file.tar > file.tar.bz2
real    838m47.771s
user    830m48.621s
sys       2m18.429s

$ time pbzip2 -c file.tar > file.tar.pbz2
real     58m06.466s
user   1748m17.785s
sys       4m49.537s

$ ls -lhgo
-rw-r--r--   1  15G Jun 24 02:03 file.tar
-rw-r--r--   1 598M Jun 24 22:10 file.tar.gz
-rw-r--r--   1 600M Jun 24 21:02 file.tar.pigz
-rw-r--r--   1 304M Jun 25 12:44 file.tar.bz2
-rw-r--r--   1 306M Jun 25 13:42 file.tar.pbz2

Hardware: Sun SPARC Enterprise T5120, 1.2GHz 8-Core SPARC V9, 4GB RAM

gzip vs. bzip2 vs. pbzip2 vs. xz vs. lzma

nospam@example.com (christian) — Sat, 28 May 2011 01:03:46 +0000

Yes, another benchmark. And yes, YMMV - big time:

$ ls -lh file.tar
-rw-r--r-- 1 bob bob 80M May 27 12:36 file.tar

$ compress-test.sh file.tar
### gzip/9c:    6 seconds / 48.700% smaller
### bzip2/9c:   44 seconds / 50.300% smaller
### pbzip2/9c:  22 seconds / 50.200% smaller
### xz/9c:      79 seconds / 53.400% smaller
### lzma/9c:    120 seconds / 53.200% smaller
### gzip/1c:    4 seconds / 46.200% smaller
### bzip2/1c:   30 seconds / 48.100% smaller
### pbzip2/1c:  15 seconds / 48.000% smaller
### xz/1c:      35 seconds / 51.100% smaller
### lzma/1c:    18 seconds / 49.700% smaller
### gzip/dc:    0 seconds
### bzip2/dc:   7 seconds
### pbzip2/dc:  4 seconds
### xz/dc:      4 seconds
### lzma/dc:    5 seconds

Versions used:

s9y testdrive - benchmarks

gzip vs. pigz vs. bzip2 vs. pbzip2

gzip vs. bzip2 vs. pbzip2 vs. xz vs. lzma