question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RPC method to translate dbt CLI strings into RPC tasks

See original GitHub issue

The rpc server provides methods like run, compile, test, etc. These methods accept a params: dictionary that configures the nature of the task. The dbt rpc server should make it possible to translate a CLI string (eg. dbt run --models abc+) into a method query. In this case, that would look like:

{
  "method": "run",
  "params": {
    "models": "abc+"
  }
}

There are broadly two ways we can accomplish this:

  1. Add a new method (eg. run_dbt_task) which accepts a CLI-string and proxies the request to the appropriate task (and returns the delegated response).
  2. Add a new method which accepts a CLI-string and returns a parsed representation of the command, which can be used to query the appropriate method

Of these, I think the first approach would be preferable (avoid an extra round-trip) if it doesn’t make the RPC server implementation too convoluted. What say you @beckjake?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
beckjakecommented, Oct 7, 2019

I’m not sure yet, but my idea is:

  • an RPC-specific argument parser that raises a dbt exception on exit and a mechanism to choose the parser class in parse_args
  • some way for RPC tasks to take either the existing dictionaries or a parsed args object
  • A new RPC task that takes in a string, builds an argparse.args, and uses the cls attribute of the parsed args to decide which “real” RPC task to delegate to

This way involves the least possible amount of code rewriting, I think.

1reaction
beckjakecommented, Oct 7, 2019

I think it’s easier to just take a string, we can feed it directly to argparse and translate the cls value into its RPC equivalent or something.

I do wish we’d had this design from the start, I would have designed it to be easier to support this behavior.

Read more comments on GitHub >

github_iconTop Results From Across the Web

RPC method to translate dbt CLI strings into RPC tasks #1811
A new RPC task that takes in a string, builds an argparse.args, and uses the cls attribute of the parsed args to decide...
Read more >
rpc | dbt Developer Hub
This server compiles and runs queries in the context of a dbt project. Additionally, the RPC server provides methods that enable you to...
Read more >
Source code for dagster_dbt.rpc.resources - Dagster Docs
[docs]class DbtRpcResource(DbtResource): """A client for a dbt RPC server. To use this as a dagster resource, we recommend using :func:`dbt_rpc_resource ...
Read more >
dbt - Database Error: 001003 (42000): SQL compilation error
I forgot that I had created a generate_schema_name macro -- and it was messing up the schema name, turning it into an empty...
Read more >
dbt Changelog - pyup.io
Fix issue when running the `deps` task after the `list` task in the RPC server ... Normalize cli-style-strings in manifest selectors dictionary ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found