Can't unzip .zip files created by OS X 10.12's native zipper

Error

On Mac OS X 10.12.6 the native zip utility (right-click “compress folder”) creates a zip file that gives this error:

extra field length exceeds extra field buffer size

Not that I don’t believe it’s possible for Apple to make a mistake, but my guess is that the underlying zip utility is probably the same Unix / Darwin zip that has been in use for a very long time and that it’s statistically more likely to be a problem with this library.

For reference, here is the failing zip file (just a couple of test files for a website):

Example Zip File

I’ve attached fails-to-parse.zip for the test case.

The contents are as follows:

.
├── css
│   ├── custom.css
│   ├── plugins
│   │   └── bootstrap.css.min
│   └── styles.css
├── index-three.html
├── index-two.html
├── index.html
└── js
    ├── app.js
    ├── config.js
    └── custom.js

I have also zipped files that do unzip just fine. My guess is that there’s some sort of off-by-one byte alignment issue.

Test Case

And a test case (based on the example):

mkdir -p /tmp/yauzl-test
pushd /tmp/yauzl-test
npm install yauzl@latest
touch test.js
wget -c https://github.com/thejoshwolfe/yauzl/files/1420073/fails-to-parse.zip
# => copy code below into test.js <= #
node test.js

test.js:

'use strict';

var yauzl = require('yauzl');

yauzl.open('./fails-to-parse.zip', function (err, zipfile) {
  if (err) throw err;

  zipfile.readEntry();
  zipfile.on('entry', function (entry) {
    if (/\/$/.test(entry.fileName)) {
      zipfile.readEntry();
      return;
    }

    zipfile.openReadStream(entry, function(err, readStream) {
      if (err) throw err;

      readStream.on("end", function() {
        zipfile.readEntry();
      });

      readStream.on("data", function (data) {
        console.log("data of length", data.length);
      });
    });
  });
  zipfile.on('error', function (err) {
    throw err;
  });
  zipfile.on('end', function () {
    console.log("all entries read and processed");
  });
});

[coolaj86]

Thank you very much. I can confirm that several files that threw errors before now parse as expected…

I’m not sure how I didn’t add the lazyEntries: true before, but I’m just grateful for your help.

[thejoshwolfe]

Ah. Sorry about that. I was so focused on the zip file that I completely neglected to look at your sample code. My bad.

I am able to reproduce the issue now. The video helped!

The problem is that you’re using both lazyEntries:false and readEntry(). You probably forgot to put {lazyEntries:true} in your yauzl.open() call.

This is such a easy mistake to make that I’m adding a check for it to throw an exception rather than this “undefined behavior”. This change should make this pitfall much more friendly to navigate. Here’s an example of what you get with this change:

$ node asdf.js 
/home/josh/dev/yauzl/index.js:231
  if (!this.lazyEntries) throw new Error("readEntry() called without lazyEntries:true");
                         ^

Error: readEntry() called without lazyEntries:true
    at ZipFile.readEntry (/home/josh/dev/yauzl/index.js:231:32)
    at /home/josh/dev/yauzl/asdf.js:8:11
    at /home/josh/dev/yauzl/index.js:36:7
    at /home/josh/dev/yauzl/index.js:137:16
    at /home/josh/dev/yauzl/index.js:623:5
    at /home/josh/dev/yauzl/node_modules/fd-slicer/index.js:32:7
    at FSReqWrap.wrapper [as oncomplete] (fs.js:629:17)

And just for reference, the {lazyEntries:true} way is the recommended way, and the only reason it’s not the default is to maintain compatibility with earlier versions of yauzl.