Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failing to decompress after strFromU8

See original GitHub issue

Issue Description

Thanks for the library! Been testing it out and ran into an interesting error:

const { strFromU8, zlibSync, strToU8, unzlibSync } = require('fflate')

const str = 'hi'
let compressed, decompressed

// This works
compressed = zlibSync(strToU8(str))
console.log(compressed)

decompressed = strFromU8(unzlibSync(compressed))
console.log(decompressed)

// This as well
compressed = strFromU8(zlibSync(strToU8(str)), true)
console.log(compressed)

decompressed = strFromU8(unzlibSync(strToU8(compressed, true)))
console.log(decompressed)

// This does not
compressed = strFromU8(zlibSync(strToU8(str)))
console.log(compressed)

decompressed = strFromU8(unzlibSync(strToU8(compressed)))
console.log(decompressed)

The output for version 0.4.1 in node.js is:

⟩ node s.js                                                                
Uint8Array(13) [
  120, 156,   1,   2, 0,
  253, 255, 104, 105, 1,
   59,   0, 210
]
hi
xýÿhi;Ò
hi
x���hi;�

/home/macobo/yyy/node_modules/fflate/lib/index.js:887
        throw 'invalid zlib data';
        ^
invalid zlib data
(Use `node --trace-uncaught ...` to show where the exception was thrown)

The same seems to occur for other compression functions as well - tested gzipSync, compressSync. Could there be a problem with strFromU8?

Issue Analytics

State:
Created 2 years ago
Comments:9 (5 by maintainers)

Top GitHub Comments

1reaction

101arrowzcommented, Oct 2, 2022

But in my testing, using strFromU8(compressionOutput, true) yields the same number of characters as strFromU8(compressionOutput) - so I’m wondering what I’m overlooking/misunderstanding.

Binary strings are really a hack to represent byte arrays with strings. The code points that map to the characters in a binary string correspond to numbers from 0-255 representing the value of the given byte. In other words, binary strings can represent arbitrary binary data.

In contrast, a normal UTF-8 string is just that - a string. It can’t represent arbitrary data, only valid Unicode characters.

So basically, if you store/transfer the string you generate with any sort of protocol that encodes strings with Unicode (HTTP, Local Storage, cookies, etc.), a binary string will take more bytes than the actual data it encodes. A binary string is really useful for compatibility with old JS where Uint8Array didn’t exist, and binary strings were the only way to represent arbitrary byte sequences. For example btoa and atob operate with binary strings, but if they were to be recreated in the modern JS landscape they’d accept ArrayBuffers instead.

0reactions

Slapboxcommented, Oct 2, 2022

Sorry for the confusion and thanks for the speedy reply @101arrowz!

In the README it says:

binary strings are incredibly inefficient and tend to double file size, so they’re not recommended.

But in my testing, using strFromU8(compressionOutput, true) yields the same number of characters as strFromU8(compressionOutput) - so I’m wondering what I’m overlooking/misunderstanding.

And I was thinking the text in this comment might be better served outside of the code sample, as it seems pretty crucial and it’s easy to overlook in the codeblock until you’re already a ways into trying to implement fflate: