ARTÍCULO
TITULO

Towards Decoupling the Selection of Compression Algorithms from Quality Constraints ? An Investigation of Lossy Compression Efficiency

Julian Martin Kunkel    
Anastasiia Novikova    
Eugen Betke    

Resumen

Data intense scientific domains use data compression to reduce the storage space needed. Lossless data compression preserves information accurately but lossy data compression can achieve much higher compression rates depending on the tolerable error margins. There are many ways of defining precision and to exploit this knowledge, therefore, the field of lossy compression is subject to active research. From the perspective of a scientist, the qualitative definition about the implied loss of data precision should only matter.With the Scientific Compression Library (SCIL), we are developing a meta-compressor that allows users to define various quantities for acceptable error and expected performance behavior. The library then picks a suitable chain of algorithms yielding the user?s requirements, the ongoing work is a preliminary stage for the design of an adaptive selector. This approach is a crucial step towards a scientifically safe use of much-needed lossy data compression, because it disentangles the tasks of determining scientific characteristics of tolerable noise, from the task of determining an optimal compression strategy. Future algorithms can be used without changing application code.In this paper, we evaluate various lossy compression algorithms for compressing different scientific datasets (Isabel, ECHAM6), and focus on the analysis of synthetically created data that serves as blueprint for many observed datasets. We also briefly describe the available quantitiesof SCIL to define data precision and introduce two efficient compression algorithms for individualdata points. This shows that the best algorithm depends on user settings and data properties.

 Artículos similares

       
 
Elisa Fiume, Gianpaolo Serino, Cristina Bignardi, Enrica Verné and Francesco Baino    
The intrinsic brittleness of bioactive glasses (BGs) is one of the main barriers to the widespread use of three-dimensional porous BG-derived bone grafts (scaffolds) in clinical practice. Among all the available strategies for improving the mechanical pr... ver más
Revista: Applied Sciences

 
Sebastian Götschel and Martin Weiser    
Solvers for partial differential equations (PDEs) are one of the cornerstones of computational science. For large problems, they involve huge amounts of data that need to be stored and transmitted on all levels of the memory hierarchy. Often, bandwidth i... ver más
Revista: Algorithms

 
Jorge Manuel Caliço Lopes de Brito, Ana Paula Gonçalves, José Roberto dos Santos     Pág. Page 33 - 40
Recycled aggregates, such as the ones used in making the sub-base and base layers of roads and the ones used as replacement of natural aggregates inconcrete production, may give an important contribution towards decreasing the negative consequences that ... ver más