Can finalize be called before entry streams are done?

Ideally, I would think I could add as many entries with streams as I want, then call finalize right afterward and everything would work (ie automatically wait for all the input streams to complete before actually creating the package). The documentation seems to imply that this isn’t the case tho. Can I or can’t i do that? If not, why not? Can we make it so finalize can be called without explicitly waiting for the streams?

[croqaz]

Hey everyone !! Any news about this issue? Adding multiple files within a tar archive would be really useful…

[abrinckm]

@croqaz @mafintosh @fresheneesz

Hi, I was able to stream multiple entries using the following code and without any modifications to mafintosh’s original source code. This will create a .tar.gz file using only streams:

const fs = require('fs');
const zlib = require('zlib');
const tar = require('tar-stream');

function exampleCreateTarGzip (files) {
    const pack = tar.pack();
    const gzipStream = zlib.createGzip({level: zlib.constants.Z_BEST_COMPRESSION});
    
    // Example list of files
    files = [ { name:"./works.csv", size: 3654 }, { name: "./manuscripts.csv", size: 303156 } ];

    return new Promise((resolve, reject) => {
      pack.on('error', reject);
      
      packEntry();
      
      // Callback is recursive
      function packEntry(err) {
        if (err) {
          reject(err);
        } else if (files.length) {
          const file_entry = files.pop();

          //please note the recursive callback here
          const entry = pack.entry({ name: file_entry.name, size: file_entry.size }, packEntry); 

          const readStream = fs.createReadStream(file_entry.name);

          readStream.on('end', entry.end.bind(entry));     // This is key to getting this to work
          readStream.on('error', reject);
          readStream.pipe(entry);
        } else {
          pack.finalize();
        }
      }

      const writeStream = fs.createWriteStream('./archive.tar.gz');
      writeStream.on('error', reject);
      writeStream.on('end', () => { resolve('Done archiving files'); });

      pack.pipe(gzipStream).pipe(writeStream);
    });
}

The key here is to call pack.entry() only after the previous entry has finished streaming and also to know when to explicitly call entry.end()

This is recursive but perhaps can be done differently. Maybe some tweaks are needed to get this to work properly for you (this was roughly copied and modified from my own code), but this solution works just fine for me.