RPC method to translate dbt CLI strings into RPC tasks
See original GitHub issueThe rpc server provides methods like run
, compile
, test
, etc. These methods accept a params:
dictionary that configures the nature of the task. The dbt rpc server should make it possible to translate a CLI string (eg. dbt run --models abc+
) into a method query. In this case, that would look like:
{
"method": "run",
"params": {
"models": "abc+"
}
}
There are broadly two ways we can accomplish this:
- Add a new method (eg.
run_dbt_task
) which accepts a CLI-string and proxies the request to the appropriate task (and returns the delegated response). - Add a new method which accepts a CLI-string and returns a parsed representation of the command, which can be used to query the appropriate method
Of these, I think the first approach would be preferable (avoid an extra round-trip) if it doesn’t make the RPC server implementation too convoluted. What say you @beckjake?
Issue Analytics
- State:
- Created 4 years ago
- Comments:7 (7 by maintainers)
Top Results From Across the Web
RPC method to translate dbt CLI strings into RPC tasks #1811
A new RPC task that takes in a string, builds an argparse.args, and uses the cls attribute of the parsed args to decide...
Read more >rpc | dbt Developer Hub
This server compiles and runs queries in the context of a dbt project. Additionally, the RPC server provides methods that enable you to...
Read more >Source code for dagster_dbt.rpc.resources - Dagster Docs
[docs]class DbtRpcResource(DbtResource): """A client for a dbt RPC server. To use this as a dagster resource, we recommend using :func:`dbt_rpc_resource ...
Read more >dbt - Database Error: 001003 (42000): SQL compilation error
I forgot that I had created a generate_schema_name macro -- and it was messing up the schema name, turning it into an empty...
Read more >dbt Changelog - pyup.io
Fix issue when running the `deps` task after the `list` task in the RPC server ... Normalize cli-style-strings in manifest selectors dictionary ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I’m not sure yet, but my idea is:
exit
and a mechanism to choose the parser class inparse_args
args
objectcls
attribute of the parsed args to decide which “real” RPC task to delegate toThis way involves the least possible amount of code rewriting, I think.
I think it’s easier to just take a string, we can feed it directly to argparse and translate the
cls
value into its RPC equivalent or something.I do wish we’d had this design from the start, I would have designed it to be easier to support this behavior.