Page History

...

The HDF5 group indicated that variable length performance would be poor. However, I see it outperform fixed size in most cases. Variable length writing is significantly faster for both chunking cases and reading 1024 element chunked data. Fixed size reading for 16 element chunks is about 10% faster than variable length.

Chunk caching

The read performance is in this usage case 20-50% less than what we see from dd (~600 MB/s). It could use some improvement.

...

where the defaults are (0,521 ,1048576 (bytes), 0.75). I For the results presented above, I  changed those parameters for the read operation to (0, 2xchunk_size, 2xchunk_size (bytes), 1).

I also ran tests to compare the speed from the increased chunk caching to that attainable with the default settings. These plots compare the reading speed from the default settings (top) to those with the increased cache size. In some cases the read speed increased (16 element fixed, 16 element variable, 1024 element variable) while decreasing for 1024 element fixed.

...