Error after downloading lots of files

I have a timer that runs every 100ms. It goes through a list of known remote file paths on the FTP server and attempts to download them to the local machine.

The timer has locks to ensure that only 3 requests are active at any one time (every time a download finishes it releases another slot to the pool).

This works well for about 20 seconds before get() errors out with

Unable to make data connection

Here is a reduced example of what is happening

function ftpController() {

    this.conn = new ftpClient();
}

ftpController.prototype.downloadFile = function(remotePath, cb) {

    cb = cb || function() {};

    localPath = mods.folderPath + remotePath;

    ftp.conn.get(remotePath, function(err, stream) {

        if (err) {

            debug('** Error downloading ' + remotePath, err);

            cb(false);
        } else {

            stream.pipe(fs.createWriteStream(localPath));
            cb(true);
        }
    });
};

// Stripped code example
ftp.downloadFile('/test/test1.txt', releaseAnotherDownload);
ftp.downloadFile('/test/test2.txt', releaseAnotherDownload);
ftp.downloadFile('/test/test3.txt', releaseAnotherDownload);

Should I be creating a new connection for each download rather than trying to use the same, is my FTP server throttling me because I’m not making a new connection for each file?

[phillipgreenii]

I was just bitten by this as well. It looks like the problem is in connection.js with self._pasvSocket = socket;. It replaces _pasvSocket each time. So for example, GET 1 sets _pasvSocket, GET 2 overrides _pasvSocket, then GET 1 completes, which clears _pasvSocket, so GET 2 fails with Unable to make data connection. In my testing, it was always the third GET that was causing the problem, but I would guess that it depends on the timing for each GET.

[rastalamm]

+1