Weird "Invalid project ID ... " error.

Environment details

• OS: Ubuntu 16.04.2 LTS, Ubuntu 16.04.4 LTS, Ubuntu 16.04.5 LTS
• Node.js version: 8.9.4, 8.11.3, 8.15.0
• npm version: 5.6.0, 6.7.0
• @google-cloud/bigquery version: 2.0.3, 2.0.6
• pm2 version: 2.10.1, 3.2.9

Description

I run node.js via pm2: pm2 restart /home/<user>/workspace/pm2.json --env development with pm2 config in json file:

{
  "apps": [
    {
      "name": "sbapi",
      "cwd": "/home/<user>/workspace/sbapi",
      "script": "./bin/www",
      "instances": 1,
      "min_uptime": "5000",
      "restart_delay": "2000",
      "exec_mode": "fork",
      "env": {
        "NODE_ENV": "development",
        "GOOGLE_APPLICATION_CREDENTIALS": "/home/<user>/workspace/sbapi/config/file_name.json"
      },
      "env_development": {
        "NODE_ENV": "development",
        "GOOGLE_APPLICATION_CREDENTIALS": "/home/<user>/workspace/sbapi/config/file_name.json"
      }
    }
  ]
}

in code with async.series I run about 38 tasks. Each send query to BigQuery:

const config = require('../../config/config');
const { BigQuery } = require('@google-cloud/bigquery');
const bigquery = new BigQuery({ projectId: config.get('googleCloud:projectId') });
const dataset = bigquery.dataset(config.get('googleCloud:bqDataset'));
const table = dataset.table(config.get('googleCloud:frontendActivitiesBQTable'));

let query = `SELECT ${queryObj.columns.join()} FROM ${config.get('googleCloud:bqDataset')}.${config.get('googleCloud:frontendActivitiesBQTable')} WHERE ${queryObj.where}`;
      if (queryObj.order) query += ` ORDER BY ${queryObj.order.join()}`;
      query += ` LIMIT 10`;
table.query(query)

as I see in console.log() my POST queries have projectId value:

{ method: 'POST',
0|sbapi  |   uri: 'https://www.googleapis.com/bigquery/v2/projects/"here I really have real project name"/jobs',
|   json: 
|    { configuration: { query: [Object] },
|      jobReference: 
|       { projectId: 'here I really have real project name',
|         jobId: '26603426-4ce9-41bd-9401-4abe9ac88555' } },
|   headers: 
|    { 'User-Agent': 'gcloud-node-bigquery/2.0.6',
|      'x-goog-api-client': 'gl-node/8.15.0 gccl/2.0.6' } }

but after about 10 good requests I get this error: body.error: { code: 400, message: ‘Invalid project ID ''. Project IDs must contain 6-63 lowercase letters, digits, or dashes. Some project IDs also include domain name separated by a colon. IDs must start with a letter and may not end with a dash.’,

   errors: 
    [ { message: 'Invalid project ID \'\'. Project IDs must contain 6-63 lowercase letters, digits, or dashes. Some project IDs also include domain name separated by a colon. IDs must start with a letter and may not end with a dash.',
        domain: 'global',
        reason: 'invalid' } ],
   status: 'INVALID_ARGUMENT' }

Maybe it’s wrong error message. P.S. It worked a month ago. Also as yurizekb wrote we can’t use bigquery normally on our project from now and project on production. It looks like some problems with API.

[yurizekb]

from https://issuetracker.google.com/issues/124026192

“After adding configuration.query.defaultDataset.projectId field to the JSON in the API explorer the error stopped showing up. This field is documented as optional (https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.query), yet it seems that some of the BigQuery nodes require it. Unfortunately the JDBC driver doesn’t set this field.”

[zsdyx]

This is also the case in Python query = """ SELECT last_modified_time FROM @project_id.@dataset_id.INFORMATION_SCHEMA.PARTITIONS WHERE table_name LIKE '@table_name%' ORDER BY last_modified_time DESC limit 1 """