"Provided schema does not match table" when running the fs-bq-schema-view script

• Extension name: firetore-bigquery-export (the gen-schema-views script)
• Extension version: firetore-bigquery-export@0.1.26, latest `gen-schema-views script´ (I´ve also tried with 0.4.2)

We’re updating our schema, and after deleting the previously generated views (as instructed by #423 and #646) running npx @firebaseextensions/fs-bq-schema-views fails with the following output:

{"code":400,"errors":[{"message":"Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)","domain":"global","reason":"invalid"}],"response":{"headers":{"alt-svc":"h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"","cache-control":"private","content-encoding":"gzip","content-type":"application/json; charset=UTF-8","date":"Wed, 16 Nov 2022 17:21:45 GMT","etag":"338qmb/WriADWLSvCyjq2w==","server":"ESF","transfer-encoding":"chunked","vary":"Origin, X-Origin, Referer","x-content-type-options":"nosniff","x-frame-options":"SAMEORIGIN","x-xss-protection":"0"}},"message":"Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)"}
Provided Schema does not match Table audit-backend:audit.audit_schema_fields_changelog. Cannot add fields (field: old_data)

The script does create the audit_schema_fields_changelog view with most of the fields (it apparently stops while trying to add the old_data field), but it does not create the other one.

I’ve updated the schema a couple of times in the past following the exact same procedure without any issues. I’m using the exact same command as before:

npx @firebaseextensions/fs-bq-schema-views \
  --non-interactive \
  --project=audit-backend \
  --dataset=audit \
  --table-name-prefix=audit \
  --schema-files=./fields.json

I’ve also explicitly set the version to the previous one (npx @firebaseextensions/fs-bq-schema-views@0.4.2 with the same result

[alapeta]

Same issue there - I have tried to upgrade extension via Firebase Console to 0.1.27 (oldData is mentioned for this release as “added feature” - whatever it is - as there are no docs) - upgrade went ok, then I deleted both views in BQ, run again npx scripts…

Unfortunately the same error

[FakhruddinAbdi]

@gustavorochakv @FakhruddinAbdi

Can you confirm that this is still an issue with the latest version.

Previously, I am assuming that you would not need to recreate the views when introducing a new field? But this is now a consistent issue?

Yes, it is still an issue with updating the schema directly It doesn’t allow to update the schema, instead, you have to remove the view manually, then run the script to recreate it