Bigquery createQueryStream broken in 5.9.1+ for large query after set options update

Environment details

• OS: Any (macOS, debian nodejs container)
• Node.js version: 14
• npm version: 8
• @google-cloud/bigquery version: 5.9.1+

Steps to reproduce

Query using createQueryStream and pass a Query object containing

• query: a large SQL string
• params: an object of params

Server replies with 400 Bad Request without many details.

Root Cause The issue was introduced in PR #999 and specifically in this commit: https://github.com/googleapis/nodejs-bigquery/pull/999/commits/48d9a1527d2a921382a9efc40a20e2fbd4c1460a

In line 2011 the whole query object is used as options which is not correct. This should be de-structured to exclude the query and params properties. I am not 100% sure what happens at the server side but I’m guessing the size of the request becomes too large when you pass the complete query object as well as the query and params properties contained within the options.

In the old version …

    queryAsStream_(query, optionsOrCallback, cb) {
        let options = typeof optionsOrCallback === 'object' ? optionsOrCallback : {};
        const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : cb;
        options = query.job
            ? extend(query, options)
            : extend(options, { autoPaginate: false });
        if (query.job) {
            query.job.getQueryResults(options, callback);
            return;
        }
        this.query(query, options, callback);
    }

In our case we are hitting the line this.query(query, options, callback) and in the old version the query param had our original query, the options param had {autoPaginate: false} and the callback was passed.

In the 5.9.1+ version when we call this.query(query, options, callback) the query param contains the original query object, the options param contains too much {autoPaginate: false, query: 'MY LONG SQL QUERY', params: {allMyParams: 'vals'} } … hence you see the problem

This is a breaking change that is preventing us from upgrading beyond 5.9.0

I can confirm that this issue does not occur when I supply a Query object containing one of our smaller SQL strings for the query property, hence why I think the 400 Bad Request from the server is due to request size.

[steffnay]

@mbyrne00 Thank you for bringing this case to my attention. I’m working on a fix now.

[mbyrne00]

Hey @steffnay - no probs, sorry. I see you’ve used that convention elsewhere too. I had troubles finding the cqs-update branch from your forked repo to look much further, so I’d say to just merge it all and I’m sure it’s fine. I’ll pull from main and if I can prove anything I’ll file a separate issue.