Databricks support

Is your feature request related to a problem? Please describe. No driver for connecting to Databricks SQL

Describe the solution you’d like I would like to work on this - adding this ticket to check noone else is already doing this or if they are to see if efforts can be combined

Describe alternatives you’ve considered Using DBeaver or other JDBC SQL clients - can’t do this for a particular client as they only want to support VSCode for their data developers.

Additional context

I read the Support new Drivers docs which were helpful, used the template https://github.com/mtxr/vsc-sqltools-driver-template

So far I’ve installed nvm, node.js, run corepack activate to get yarn ( had many issues trying to use npm and I notice this main project is using yarn so going with that) edited the package.json, and trying to build/debug.

I am working through the required hooks in the src/extension.js file, referring to existing drivers as examples but instead using the databricks-sql-nodejs driver.

Some additional features I’d like to have would be using the same dotfile/profile for connection string details that the databricks CLI uses ( btw the postgreSQL driver could also benefit from preferencing the ~/.pg_service.conf file for connection details)

I also see other drivers have issues with too many rows returned and suggest users always append LIMIT 1000 - I would like to always wrap queries with that. And also consider the : for delimiting multiple queries in some way.

[gjsjohnmurray]

How is your driver development going? If / when it is published let us know and we’ll add a link to it in the README and online documentation of the main extension.

[davehowell]

Hi @KaduUlson I haven’t published it, I think that without the metadata tree working it’s a bit too raw. I am wondering, do you use “Unity Catalog”? Adding support just for Unity Catalog might be an easier short-term way to get it to a publishable state. TBH I’d prefer not to support the legacy hive metastore catalog, mostly because the API for it is painful and slow. Unity Catalog has the ISO/ANSI standard “information schema” that the SQL Tools core already supports. I will revisit it in the next few days.