Unhelpful `PartialFailureError` when doing a raw insert.

Having trouble doing a “raw” insert, as described in the API docs here: https://googleapis.dev/nodejs/bigquery/latest/Table.html#insert

It fails with PartialFailureError. Digging into the errors array on it, there’s one element: { message: '', reason: 'invalid' }, which doesn’t give me much to go by.

The operation itself is super simple. It’s simply inserting a string into a table with one column (string).

Environment details

• OS: MacOS
• Node.js version: 8 (Cloud Functions engine version)
• npm version: 6.13.6
• @google-cloud/bigquery version: 4.6.1

Steps to reproduce

Doing a table.insert like so:

const bq = new BigQuery({ location: 'EU' });
const dataset = bq.dataset('firestore_export');
const table = dataset.table('test');

table.insert({
   skipInvalidRows: false,
   ignoreUnkownValues: false,
   rows: [
      {
        insertId: event.eventId, // (comes from the CF trigger's context.eventId)
        json: {
          document_name: 'test',
        }
      }
   ],
}, { raw: true });

My code is based this, which seems to work fine: https://github.com/firebase/extensions/blob/next/firestore-bigquery-export/firestore-bigquery-change-tracker/src/bigquery/index.ts

Anyway, is PartialFailureError not supposed to give a bit more info about what went wrong?

[shollyman]

Essentially, this error response is informing you that the the row data you’ve sent is invalid (it doesn’t conform to what the streaming API understands to be the current schema of the table).

You’ve disabled skipInvalidRows, so the system is surfacing this error.

It may be the case that you’re constructing incompatible messages, or it may be a case of a table has recently changed (deleted/recreated, schema change, etc) and the streaming system hasn’t yet picked up the change. More info here: https://cloud.google.com/bigquery/docs/error-messages#streaming

There’s nothing really actionable here beyond that from the library perspective… The API will not provide more expansive responses describing what’s wrong.

Closing this one for now.

[larssn]

No, I dont use raw inserts as a result