firestore-bigquery-export fs-bq-import-collection doesn't allow {widecard} path
See original GitHub issue[READ] Step 1: Are you in the right place?
Issues filed here should be about bugs for a specific extension in this repository. If you have a general question, need help debugging, or fall into some other category use one of these other channels:
- For general technical questions, post a question on StackOverflow with the firebase tag.
- For general Firebase discussion, use the firebase-talk google group.
- To file a bug against the Firebase Extensions platform, or for an issue affecting multiple extensions, please reach out to Firebase support directly.
[REQUIRED] Step 2: Describe your configuration
- Extension name: firestore-bigquery-export
- Configuration values (redact info where appropriate):
- fs-bq-import-collection
[REQUIRED] Step 3: Describe the problem
Steps to reproduce:
- Clone this repo.
- Go to extensions/firestore-bigquery-export/scripts/import and install dependencies (npm install)
- Execute
npm run import
- Pass the ${PROJECT_ID}
- Try pass a widecard path as described on docs, like
chats/{chatid}/posts
- Get an error:
>> The collection path must only contain letters or spaces
Expected result
Allow widecard paths to import.
Actual result
Get an error: >> The collection path must only contain letters or spaces
Issue Analytics
- State:
- Created 4 years ago
- Reactions:23
- Comments:24 (5 by maintainers)
Top Results From Across the Web
Loading data from Firestore exports | BigQuery - Google Cloud
BigQuery supports loading data from Firestore exports created using the Firestore managed import and export service. The managed import and export service ...
Read more >How to configure the “Export Collections to BigQuery ...
Let's use the “QUERY” button at the top to query this BigTable view. A SELECT * FROM SQL query returns two rows, corresponding...
Read more >Existing firestore collection data into bigQuery - Stack Overflow
This extension only sends the content of documents that have been changed -- it does not export your full dataset of existing documents...
Read more >Re: Firestore -> BigQuery - Google Cloud Community
The process of exporting your data and updates in real-time is called streaming and luckily, there is an extension that helps you stream...
Read more >Stream Collections to BigQuery - Extensions Marketplace
Use this extension to export the documents in a Cloud Firestore collection to ... it does not export your full dataset of existing...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@kienner-philippe and @garciapd, I was able to get around this by making my own copy of the script and modifying the query to use collectionGroups.
Changed this: https://github.com/firebase/extensions/blob/next/firestore-bigquery-export/scripts/import/src/index.ts#L188
To this:
Added these to my package.json:
Then I just run my local version of the script using
ts-node
with something like this:when the script asks:
Give it the sub-collection name for the collection group query.
posts
please support sub collections!!