Unrecoverable failure in Jinja templating: 'dbt_utils' is undefined

I’m trying to run sqlfluff on a dbt model:

{{ config(materialized='view') }}

select
    {{ dbt_utils.hash("message_id") }},
    *
from user_session_events

But I received the following error:

L:   - | P:   - |  TMP | Unrecoverable failure in Jinja templating: 'dbt_utils'
                       | is undefined. Have you configured your variables?
                       | https://docs.sqlfluff.com/en/latest/configuration.html
L:   4 | P:   8 |  TMP | Undefined jinja template variable: 'dbt_utils'

I’ve found the section about updating the Jinja templator by using load_macros_from_path in the .sqlfluff file.

So I tried updating my config file by adding the install location of dbt_util’s macros directory:

[sqlfluff:templater:jinja]
apply_dbt_builtins = True
load_macros_from_path=./dbt_modules/dbt_utils/macros

But now when I run sqlfluff lint, I receive an error saying that sqlfluff can no longer parse the materialization tag:

Traceback (most recent call last):
  File "/usr/local/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/cli/commands.py", line 172, in lint
    result = lnt.lint_paths(paths, verbosity=verbose, ignore_non_existent_files=False)
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/linter.py", line 912, in lint_paths
    ignore_non_existent_files=ignore_non_existent_files))
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/linter.py", line 898, in lint_path
    fix=fix, config=config))
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/linter.py", line 713, in lint_string
    parsed, vs, time_dict = self.parse_string(s=s, fname=fname, verbosity=verbosity, config=config)
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/linter.py", line 611, in parse_string
    s, templater_violations = self.templater.process(s, fname=fname, config=config or self.config)
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/templaters.py", line 319, in process
    ctx.update(self._extract_macros_from_path(macros_path, env=env, ctx=live_context))
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/templaters.py", line 210, in _extract_macros_from_path
    env=env, ctx=ctx
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/templaters.py", line 200, in _extract_macros_from_path
    template, env=env, ctx=ctx
  File "/usr/local/lib/python3.7/site-packages/sqlfluff/templaters.py", line 175, in _extract_macros_from_template
    macro_template = env.from_string(template, globals=ctx)
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 941, in from_string
    return cls.from_code(self, self.compile(source), globals, None)
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 638, in compile
    self.handle_exception(source=source_hint)
  File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "<unknown>", line 53, in template
jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'materialization'.

Version

sqlfluff, version 0.3.6 Python 3.7.8

Configuration

[sqlfluff]
dialect = snowflake

[sqlfluff:templater:jinja]
apply_dbt_builtins = True
load_macros_from_path=./dbt_modules/dbt_utils/macros

[akcx]

I am facing the same issue for the dbt builtin var.

I have apply_dbt_builtins enabled in my config.

[sqlfluff:templater:jinja]
apply_dbt_builtins = True
load_macros_from_path = macros

I get the following error on running lint.

L:   1 | P:   1 |  TMP | Unrecoverable failure in Jinja templating: 'var' is
                       | undefined. Have you configured your variables?
                       | https://docs.sqlfluff.com/en/latest/configuration.html

I also tried using using custom macros defined under [sqlfluff:templater:jinja:macros] based on instructions on this page but it didn’t work for me.

I am running sqlfluff version 0.11.0.

Could anyone please help me understand what the problem is?

Thanks!

[NiallRees]

Hi @tjwaterman99 thanks for this - I think your issue will be solved in 0.4.0 (latest pre release currently 0.4.0a3) with the addition of dbt templating. See https://docs.sqlfluff.com/en/latest/configuration.html?highlight=dbt#dbt-project-configuration for guidance and feel free to drop any questions you may have in the Slack workspace, there’s an invite link in the readme if you’ve not joined already. Going to close this for now 🙂