Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

dbt.exceptions.InternalException: Asked to compile <class 'dbt.contracts.graph.compiled.CompiledModelNode'> node, but it has no compiled form

See original GitHub issue

Looks like we’re seeing identical behavior to #1518 , #1613 , and #857 after upgrading sqlfluff to version 0.7.1. When attempting to run sqlfluff lint on our dbt project, we get the following exception:

dbt@b263229e2f53:/workspaces/dbt-dutchie$ sqlfluff lint models/shared/dispensaries.sql models/shared/intermediate/int__dispensaries.sql  --nofail
=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
=== [dbt templater] Sorting Nodes...
Traceback (most recent call last):
  File "/opt/venv/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/cli/commands.py", line 389, in lint
    result = lnt.lint_paths(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 851, in lint_paths
    self.lint_path(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 825, in lint_path
    for linted_file in runner.run(fnames, fix):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 95, in run
    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 48, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 41, in iter_rendered
    yield fname, self.linter.render_file(fname, self.config)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 576, in render_file
    return self.render_string(raw_file, fname, config, encoding)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 553, in render_string
    templated_file, templater_violations = self.templater.process(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 308, in process
    processed_result = self._unsafe_process(fname_absolute_path, in_str, config)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 377, in _unsafe_process
    node = self.dbt_compiler.compile_node(
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 543, in compile_node
    node = self._compile_node(node, manifest, extra_context)
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 377, in _compile_node
    compiled_node = _compiled_type_for(node).from_dict(data)
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 39, in _compiled_type_for
    raise InternalException(
dbt.exceptions.InternalException: Asked to compile <class 'dbt.contracts.graph.compiled.CompiledModelNode'> node, but it has no compiled form

Expected Behaviour

No exceptions.

Observed Behaviour

dbt@b263229e2f53:/workspaces/dbt-dutchie$ sqlfluff lint models/shared/dispensaries.sql models/shared/intermediate/int__dispensaries.sql  --nofail
=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
=== [dbt templater] Sorting Nodes...
Traceback (most recent call last):
  File "/opt/venv/bin/sqlfluff", line 8, in <module>
    sys.exit(cli())
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/cli/commands.py", line 389, in lint
    result = lnt.lint_paths(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 851, in lint_paths
    self.lint_path(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 825, in lint_path
    for linted_file in runner.run(fnames, fix):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 95, in run
    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 48, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/runner.py", line 41, in iter_rendered
    yield fname, self.linter.render_file(fname, self.config)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 576, in render_file
    return self.render_string(raw_file, fname, config, encoding)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff/core/linter/linter.py", line 553, in render_string
    templated_file, templater_violations = self.templater.process(
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 308, in process
    processed_result = self._unsafe_process(fname_absolute_path, in_str, config)
  File "/opt/venv/lib/python3.9/site-packages/sqlfluff_templater_dbt/templater.py", line 377, in _unsafe_process
    node = self.dbt_compiler.compile_node(
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 543, in compile_node
    node = self._compile_node(node, manifest, extra_context)
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 377, in _compile_node
    compiled_node = _compiled_type_for(node).from_dict(data)
  File "/opt/venv/lib/python3.9/site-packages/dbt/compilation.py", line 39, in _compiled_type_for
    raise InternalException(
dbt.exceptions.InternalException: Asked to compile <class 'dbt.contracts.graph.compiled.CompiledModelNode'> node, but it has no compiled form

Steps to Reproduce

sqlfluff lint any files in the dbt project.

Dialect

snowflake

Version

$ sqlfluff --version
sqlfluff, version 0.7.1
$ python --version
Python 3.9.7
$ poetry show sqlfluff
name         : sqlfluff
version      : 0.7.1
description  : The SQL Linter for Humans

dependencies
 - appdirs *
 - cached-property *
 - click >=7.1
 - colorama >=0.3
 - configparser *
 - diff-cover >=2.5.0
 - Jinja2 *
 - oyaml *
 - pathspec *
 - pytest *
 - tblib *
 - toml *
 - typing-extensions *
$ poetry show sqlfluff-templater-dbt
name         : sqlfluff-templater-dbt
version      : 0.7.1
description  : Lint your dbt project SQL.

dependencies
 - dbt >=0.17
 - sqlfluff >=0.7.0

Configuration

[sqlfluff]
dialect = snowflake
templater = dbt
rules = L001,L002,L003,L004,L005,L006
ignore = parsing,templating

[sqlfluff:rules]
max_line_length = 120
comma_style = trailing

[sqlfluff:rules:L010]
capitalisation_policy = upper

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
barrywhartcommented, Feb 14, 2022

Reopening this, because I realize we should look at whether the current behavior causes issues with pre-commit, where it attempts to check and fix all modified files. Ephemeral models are located in the same directory as normal models, so pre-commit will probably try and lint them whether ephemeral or not.

I don’t think SQLFluff can actually lint these files, but it could likely handle the situation better:

  • Print a reasonable warning or error message
  • Don’t return a failure code, because this in turn causes pre-commit to block committing

Could we (should we) allow working around this with “noqa”? Or just handle this automatically? 🤔

0reactions
barrywhartcommented, Mar 1, 2022

Closing this issue after one month of no feedback. We’ve made some fixes to handling ephemeral models now. PR #2748 adds a dbt test case where it compiles 3 ephemeral models A, B, C, where A depends on B, which depends on C, so it is possible to lint models that are purely ephemeral.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Compiling models will break runs for different models
I have all my run jobs to execute particular groups of models, all sharing the same directory. I thought that by using dbt...
Read more >
How to use the dbt.node_types.NodeType function in dbt
To help you get started, we've selected a few dbt.node_types.NodeType examples, based on popular ways it is used in public projects.
Read more >
Configuration — SQLFluff 1.4.5 documentation
Library Templating¶. If using SQLFluff for dbt with jinja as your templater, you may have library function calls within your sql files that...
Read more >
Source code for dagster_dbt.asset_defs
try: import dbt.graph.cli as graph_cli import dbt.graph.selector as ... some nodes exist inside the dbt graph but are not assets resource_type ...
Read more >
Graph Query Portal - VTechWorks
with creating a graph representation of the data collected from the social ... wrong. They also learned how to work with Neo4j and...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

L042 loop limit on fixes reached when CTE itself contains a subquery

Next page

dbt templater mangles multi-byte characters after `sqlfluff fix`