question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Make BigQuery cache lookups case-insensitive

See original GitHub issue

UPDATE: added step 4 to reproduction recipe

Describe the bug

The is_incremental() macro always returns false when the target is BigQuery, effectively disabling the incremental build feature.

Steps To Reproduce

  1. Create a fresh DBT project with dbt init
  2. Configure the profile to use BigQuery
  3. Change the “my_first_dbt_model.sql” file to this:

{{ config(materialized=‘incremental’) }} select 1 as id {% if is_incremental() %} THIS SHOULD CAUSE AN ERROR {% endif %}

  1. execute “dbt run” to initially create the table
  2. execute “dbt run” again => no error is displayed (bug)
  3. execute “dbt run --bypass-cache” => error is displayed (expected behaviour)

Expected behavior

An error message should be displayed when calling “dbt run” because the SQL inside the is_incremental condition is invalid. When running “dbt run --bypass-cache” the result is as expected.

Additional information

I did a little debugging and verified that is_incremental is indeed returning false. The cause is that adapter.get_relation apparently returns None. Since the problem vanishes when --bypass-cache is specified, I suspect that the problem is caused by a bug in the relation cache.

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • [X ] bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.14.2
   latest version: 0.14.2

Up to date!

The operating system you’re using: macOS Mojave

The output of python --version: Python 3.7.4

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
haukedudencommented, Oct 7, 2019

I just tried again with a lower-case dataset name. Then I also get the expected error message on the second call. So it seems that this might indeed be caused by upper characters in the original dataset name.

0reactions
drewbanincommented, Oct 7, 2019

Thanks @haukeduden - that’s super helpful. I appreciate you taking the time to report this one – we’ll take a look at fixing it!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Make BigQuery cache lookups case-insensitive #1810 - GitHub
Describe the bug. The is_incremental() macro always returns false when the target is BigQuery, effectively disabling the incremental build ...
Read more >
Using cached query results | BigQuery - Google Cloud
BigQuery writes all query results to a table. The table is either explicitly identified by the user (a destination table), or it is...
Read more >
Case insensitive LIKE in BigQuery - sql - Stack Overflow
Is there any way of using case insensitive LIKE, e.g. LIKE device_type LIKE '%iphone 7%' in BigQuery ? Google Cloud Collective.
Read more >
14 Best Practices to Tune BigQuery SQL Performance
#14: Take advantage of Caching. Note 1: All queries here are written based on the BigQuery public data, which is accessible by everyone....
Read more >
Devart PostgreSQL Lookup
The number of the queried rows from the lookup object to cache. Default value is 2000. CaseInsensitiveLookup. Determines whether matching is case insensitive....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found