...

SZ compressor from Argonne

https://github.com/disheng222/SZ

-> Clone or download -> Download ZIP -> installed under ~/lib/sz/sz-1.4.9/

Run tests like:

~/lib/sz/sz-1.4.9/SZ-master/example]$ ./testfloat_compress sz.config testdata/x86/testfloat_8_8_128.dat 8 8 128

• works with float and double.
• int16 and uint16 not implemented

compression factors ~ 56, 110, and 49 for  
- testfloat_8_8_128.dat,
- testdouble_8_8_128.dat, and 
- testdouble_8_8_8_128.dat, respectively.
 

But for data with VERY NARROW SPECTRA:

 testfloat_8_8_128.txt            mean=1.000000  std=1.232407
testdouble_8_8_128.txt       mean=1.000000  std=1.254261
testdouble_8_8_8_128.txt  mean=1.300935   std=0.502083 

References

• Using compression in HDF5
• Szip Compression in HDF
• Third-party compression filters
• HDF5 Tutorial
• HDF5 Software Documentation
• Using HDF5 filters
• HDF5 Data Compression Demystified
• gzip - CLI, zlib - API
• Entropy (information_theory)
• Dictionary compression
• Python and HDF5
• LibLZF
• SZ compressor (sz-1.4-user-guide.pdf) Authors: Sheng Di, Dingwen Tao, supervisor: Franck CappelloCappello 
• 2017-02-24-Report-of-SZ-Lossy-Compression-on-EXAFEL-Datasets-v5.pdf
• 2013-04-17-igor-pyana_xtc_decompression_status.pdf, psdmrepo
• 2017-03-21-Lossless-compression.pdf
• 2017-09-27-Lossless-compression.pdf - on Users mtg