question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

bigquery-export: Add `document.id` to BigQuery tables

See original GitHub issue

[REQUIRED] Step 2: Extension name

This feature request is for extension: firestore-bigquery-export

What feature would you like to see?

For this feature to be useful to me, I really need the firestore document’s id exposed as a single field in the generated tables. Currently, the fully qualified document_name is provided, but this includes a bunch of additional prefixes that makes it really difficult/cumbersome to join across tables.

How would you use it?

I want to use the big query tables to power DataStudio as well as for running one-off queries to better understand my app’s usage.

I often store data semi-normalized using document identifiers, such as memberId (which is like a userId), and then need to join to the memebers collection, using memberId as the join key.

Currently, I would have to concact a string together to join on document_name in this format

projects/XX-project-nameXX/databases/(default)/documents/members/<my member document id>

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
russellwheatleycommented, Jul 23, 2020

That would be my recommendation 😄 .

1reaction
russellwheatleycommented, Jul 23, 2020

@mustafaekim May I ask what version of the firestore-bigquery-export extension you’re using? You should be able to see it in your Firebase console dashboard in the extension’s tab.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Exporting table data | BigQuery - Google Cloud
Describes how to export data from BigQuery to Cloud Storage in CSV, JSON, Avro, and Parquet formats.
Read more >
How to Export Data to a File in Google BigQuery - Chartio
Go to the BigQuery WebUI. · Select the table you wish to export. · Click on Export Table in the top-right. · Select...
Read more >
Export Firebase Crashlytics data to BigQuery
Requires enabling Crashlytics BigQuery streaming export. In this example, we combine batch and realtime tables to add realtime information to the reliable batch ......
Read more >
Stream Collections to BigQuery - Extensions Marketplace
Use this extension to export the documents in a Cloud Firestore collection to BigQuery. Exports are realtime and incremental, so the data in...
Read more >
Re: Firestore -> BigQuery - Google Cloud Community
In order to stream Firestore data to BigQuery and update your BigQuery data when a document or a collection changes, you can use...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found