question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Delta Live Tables

See original GitHub issue

Describe the feature

Create and orchestrate Delta Live Tables using dbt-databricks

Describe alternatives you’ve considered

Delta Live Tables can be set up using the Databricks UI, but this misses all the excellent features of dbt.

Additional context

I do not know whether this idea is feasible, since Delta Live Tables function only in tandem with Databricks pipelines, which goes beyond CREATE TABLE.

Who will this benefit?

Anyone who likes both dbt and Delta Live Tables, in particular teams with existing dbt-databricks projects who want to migrate to Delta Live Tables.

Are you interested in contributing this feature?

Potentially, but I am a complete newbie to this repo.

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:12
  • Comments:5

github_iconTop GitHub Comments

4reactions
Angelo-Barufficommented, Oct 18, 2022

@bilalaslamseattle the ideia is to use all the power of dbt framework, like tests, models, code structure, jinja, dbt utils, etc… and combine with the power of DLT using streaming data easily. As @dwolfeu said, its a good integration for teams that already work with dbt

1reaction
bilalaslamseattlecommented, Oct 19, 2022

@Angelo-Baruffi makes a lot of sense. Thank you!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Delta Live Tables - Databricks
Delta Live Tables (DLT) makes it easy to build and manage reliable batch and streaming data pipelines that deliver high-quality data on the...
Read more >
Delta Live Tables introduction - Azure Databricks
Delta Live Tables is a framework for building reliable, maintainable, and testable data processing pipelines. You define the transformations ...
Read more >
databricks/delta-live-tables-notebooks - GitHub
Delta Live Tables is a new framework designed to enable customers to successfully declaratively define, deploy, test & upgrade data pipelines and eliminate ......
Read more >
Why the Databricks Delta Live tables are the next big thing?
Delta Live Table is a simple way to build and manage data pipelines for fresh, high-quality data. It provides these capabilities: Easy pipeline ......
Read more >
Delta Live Tables with Auto Loader and Hugging ... - YouTube
Modern data engineering requires more advanced data lifecycle for data ingestion, transformation, and processing. In this session, learn how ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found