DEV Community

Vipertech.ch
Vipertech.ch

Posted on

Compress any data in JS with LZP3 (very powerful algorithm).

Example use-case

In the https://pixa.pics/ project, we enable people to draw pixel arts into a 2d canvas, it must store the image source, all modification for undo/redo (our history system is dumb, we don't store reverse operations rather than the whole change...) and the thumbnail with metadata, it auto-save a compressed blob into the browser and it is quite fast within a web worker, it doesn't freeze the UI.

Compression benchmarks

Here are some representative speeds and sizes for the various algorithms implemented in the original package (https://github.com/eladkarako/compressjs-flattened). Times are with node 0.8.22 on my laptop, but they should be valid for inter-algorithm comparisons.

Text / Test

This is the Taoism article from the Simple English wikipedia, in HTML format as generated by the Wikipedia Parsoid project.

Type Level Size (bytes) Compress time (s) Decompress time (s)
bwtc 9 272997 13.10 1.85
bzip2 9 275087 22.57 1.21
lzp3 - 292978 1.73 1.74
ppm - 297220 42.05 44.04
bzip2 1 341615 22.63 1.40
bwtc 1 345764 12.34 0.80
dmc - 434182 6.97 9.00
lzjbr 9 491476 3.19 1.92
lzjbr 1 523780 2.76 2.02
lzjb 9 706210 1.02 0.30
lzjb 1 758467 0.66 0.29
context1 - 939098 5.20 4.69
fenwick - 1440645 3.06 3.72
mtf - 1441763 1.92 3.86
huffman - 1452055 7.15 6.56
simple - 1479143 0.72 2.42
defsum - 1491107 3.19 1.46
no - 2130648 0.80 0.92
- - 2130640 - -
var func = async function(uint8a_or_obj, mode) {return new Promise(function(resolve, reject){
...
if(mode === "COMPRESS_OBJECT") {
    //  JS -> json_str -> ui8a -> compressed ui8a
    resolve(Uint8Array.from(Lzp3.compressFile(new TextEncoder("utf-8").encode(JSON.stringify(uint8a_or_obj, null, 0)))));

}else if(mode === "DECOMPRESS_UINT8A") {
    // ui8a decompressed -> ui8a -> json_str -> JS
    resolve(JSON.parse(new TextDecoder().decode(Uint8Array.from(Lzp3.decompressFile(uint8a_or_obj)))));

}else {

    reject(null);     
}
})};
Enter fullscreen mode Exit fullscreen mode

Real implementation: https://github.com/pixa-pics/pixa-pics.github.io/blob/main/src/js/utils/lzp3_json.js

var blob = new Blob([uint8a], {type : "application/octet-stream"}); /* ... */
blob.arrayBuffer().then((array_buffer) => {var uint8a = new Uint8Array(array_buffer); /* ... */ });
Enter fullscreen mode Exit fullscreen mode

Discussion (0)