Create bqutil UDFs in all other non-US datasets
See original GitHub issueI ran into some issues when trying to use CTE’s in combination with bqutil.
This exectues as expected:
SELECT `bqutil.fn.median`([1,1,1,2,3,4,5,100,1000]) as median
However, after adding a CTE:
WITH covid AS (
SELECT date, daily_confirmed_cases
FROM `bigquery-public-data.covid19_ecdc_eu.covid_19_geographic_distribution_worldwide`
)
SELECT `bqutil.fn.median`([1,1,1,2,3,4,5,100,1000]) as median
BQ throws the error: “Function not found: bqutil.fn.median
”.
I there a way to explicitly import the BQ utils or any other suggestions to address this issue?
Issue Analytics
- State:
- Created 2 years ago
- Comments:6
Top Results From Across the Web
User-defined functions | BigQuery - Google Cloud
A user-defined function (UDF) lets you create a function by using a SQL expression or JavaScript code. A UDF accepts columns of input,...
Read more >BigQuery UDFs Complete Guide - Towards Data Science
Everything you need to know about Google BigQuery User-Defined Functions ... We will use plain old SQL to define the functions with the...
Read more >bigquery-utils/README.md at master - udfs - GitHub
All UDFs within this repository are available under the bqutil project on publicly shared datasets. Queries can then reference the shared UDFs via...
Read more >Support for BigQuery User-Defined Functions (UDFs) #1289
I do get query validation warnings: "Some dependencies don't yet exist in the warehouse". It seems like Dataform doesn't know that the functions ......
Read more >New in BigQuery: Persistent UDFs - Felipe Hoffa - Medium
Create org-wide libraries of business logic within shared datasets. For example, now anyone can call this number parsing function: SELECT fhoffa ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@danieldeleo Apologies for commenting on this closed issue, do you anticipate that bqutil be publicly available in non-US locations or if we are outside the US should we always deploy our own? Thanks
It’s not clear from the error message, but this is caused by the table and the UDF being in different BigQuery dataset locations, and the query must be processed in only one location. By default, BigQuery will attempt to process your query in EU location since
bigquery-public-data.covid19_ecdc_eu.covid_19_geographic_distribution_worldwide
is stored in EU. The bqutil.fn dataset is only available in US location for now, so the query errors because it cannot find the UDF in the EU location. For now, you can create the UDF as either a persistent UDF in your own dataset, or as a temporary UDF in a script along with your query.I will assign this issue to myself to have all the bqutil UDFs created in the other non-US locations