Readonly Databases not Supported (eg Dremio or lower permissioned connections)

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if any
• I have reproduced the issue with at least the latest released version of superset
• I have checked the issue tracker for the same issue and I haven’t found one similar

Superset version

0.24

Expected results

I should be able to add a table from any database connection. Note that when I add more database connections I can use them just fine in the SQL Lab UI - just can’t add tables from them in the Tables function - so I can’t actually use them in charts or dashboards.

It seems like the Superset system is only designed to use a single database at a time and requires read/write access to that database, which would block my planned use of the system where I planned to have a small postgresql database for metadata and management and a large readonly database (Dremio in my case) as the analytical datastore.

Actual results

When adding a new Table item in the UI the system gets confused and queries the target database to check if the “tables” table already contains the new entry - rather than checking the main database connection (sqlite in my case). The system returns an Unknown error from database Flash message and the query itself fails in the Superset system logs (table “Tables” does not exist)

Steps to reproduce

Install the system as is with the Sqlite datasource/database as the primary database (load samples etc). Then Add a new Database in addition to the SQLLite one. In my case add a database where new tables cannot be created (although this seems to be beside the point). Attempt to add a new Table that is known to exist in the secondary database. Watch the system fail to add the new table (generating the unknown database flash message)

Appears to be a problem in (or leading up to)

superset/connectors/sqla/views.py line 236 pre_add method 

This seems to grab the wrong database sqla engine value and implications are that the system needs to know which is the database engine that holds the Superset metadata and which is the system where the new table is to be added from.

SQL Trace

2018-04-30 20:14:20,274 INFO sqlalchemy.engine.base.Engine SELECT EXISTS (SELECT 1 
FROM tables 
WHERE tables.table_name = ? AND tables.schema = ? AND tables.database_id = ?) AS anon_1
2018-04-30 20:14:20,274:INFO:sqlalchemy.engine.base.Engine:SELECT EXISTS (SELECT 1 
FROM tables 
WHERE tables.table_name = ? AND tables.schema = ? AND tables.database_id = ?) AS anon_1
2018-04-30 20:14:20,275 INFO sqlalchemy.engine.base.Engine (u'sf_incidents_clean', u'', 3)
2018-04-30 20:14:20,275:INFO:sqlalchemy.engine.base.Engine:(u'sf_incidents_clean', u'', 3)
2018-04-30 20:14:20,275:INFO:root:Database.get_sqla_engine(). Masked URL: dremio+pyodbc://malsmith:XXXXXXXXXX@mydremio
2018-04-30 20:14:20,562 INFO sqlalchemy.engine.base.Engine ROLLBACK
2018-04-30 20:14:20,562:INFO:sqlalchemy.engine.base.Engine:ROLLBACK
2018-04-30 20:14:20,564:INFO:werkzeug:127.0.0.1 - - [30/Apr/2018 20:14:20] "POST /tablemodelview/add HTTP/1.1" 302 -

[tomasienrbc]

@mistercrunch thanks Maxime! I’m going to try to get the SQL Alchemy dialect working, the team that built it abandoned the project BUT I think left it in a working or semi working state. If anyone else would rather try it out please let me know, I’m only able to work on this on the weekends but if I get it working I’ll @ everyone who was interested here

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.