Unable to use cockroachdb in apache superset.

Describe the problem

Unable to use cockroachdb in Apache Superset as data souce.

To Reproduce

-> install and setup superset -> login -> create new source -> put cockroach db source stringas sqlalchemy URI -> test the connection and save it as data source. -> import table definition -> select cockroach db as source and table name and save

What did you do? Describe in your own words.

Apache’s incubator project, Apache Superset is a platform for data visualisations. It is a python application. The app is using SQLAlchemy ORM. By installing the python pip package for SQLAlchemy dialect for cockroachdb, I was able to save cockroach db connection as data source. While trying to create table definition out of that connection, it throws an error saying table [tableName] could not be found. Please double check database, schema and table.

Superset version

Superset version: 0.28.1 CochroachDB version: 2.1.8 cockroachdb-python version: 0.2.1

Expected results

Should be able to save table definition successfully.

Actual results

table [tableName] could not be found. Please double check database, schema and table.

Steps to reproduce

-> create new source -> put cockroach db source stringas sqlalchemy URI -> test the connection and save it as data source. -> import table definition -> select cockroach db as source and table name and save

Additional data / screenshots

stacktrace

1.first stacktrace

Traceback (most recent call last): File “/venv/local/lib/python3.6/site-packages/superset/connectors/sqla/views.py”, line 256, in pre_add table.get_sqla_table_object() File “/venv/local/lib/python3.6/site-packages/superset/connectors/sqla/models.py”, line 804, in get_sqla_table_object return self.database.get_table(self.table_name, schema=self.schema) File “/venv/local/lib/python3.6/site-packages/superset/models/core.py”, line 931, in get_table autoload_with=self.get_sqla_engine()) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 457, in new metadata._remove_table(name, schema) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py”, line 66, in exit compat.reraise(exc_type, exc_value, exc_tb) File “/WIP/venv/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py”, line 249, in reraise raise value File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 452, in new table._init(name, metadata, *args, **kw) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 534, in _init include_columns, _extend_on=_extend_on) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 547, in _autoload extend_on=extend_on File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2056, in run_callable return conn.run_callable(callable, *args, **kwargs) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 1545, in run_callable return callable(self, *args, **kwargs) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py”, line 391, in reflecttable table, include_columns, exclude_columns, **opts) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py”, line 618, in reflecttable table_name, schema, **table.dialect_kwargs): File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py”, line 369, in get_columns **kw) File “/venv/local/lib/python3.6/site-packages/cockroachdb/sqlalchemy/dialect.py”, line 151, in get_columns typ = type_class() TypeError: ‘NullType’ object is not callable

1. second one

Traceback (most recent call last): File “/venv/local/lib/python3.6/site-packages/superset/connectors/sqla/views.py”, line 256, in pre_add table.get_sqla_table_object() File “/venv/local/lib/python3.6/site-packages/superset/connectors/sqla/models.py”, line 804, in get_sqla_table_object return self.database.get_table(self.table_name, schema=self.schema) File “/venv/local/lib/python3.6/site-packages/superset/models/core.py”, line 931, in get_table autoload_with=self.get_sqla_engine()) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 457, in new metadata._remove_table(name, schema) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py”, line 66, in exit compat.reraise(exc_type, exc_value, exc_tb) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py”, line 249, in reraise raise value File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 452, in new table._init(name, metadata, *args, **kw) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 534, in _init include_columns, _extend_on=_extend_on) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py”, line 547, in autoload extend_on=extend_on File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 2056, in run_callable return conn.run_callable(callable, *args, **kwargs) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py”, line 1545, in run_callable return callable(self, *args, **kwargs) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py”, line 391, in reflecttable table, include_columns, exclude_columns, **opts) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py”, line 618, in reflecttable table_name, schema, **table.dialect_kwargs): File “/venv/local/lib/python3.6/site-packages/sqlalchemy/engine/reflection.py”, line 369, in get_columns **kw) File “/venv/local/lib/python3.6/site-packages/cockroachdb/sqlalchemy/dialect.py”, line 152, in get_columns print(type_class) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/type_api.py”, line 589, in str return str(self.compile()) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/type_api.py”, line 573, in compile return dialect.type_compiler.process(self) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/compiler.py”, line 293, in process return type._compiler_dispatch(self, **kw) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py”, line 81, in compiler_dispatch return meth(self, **kw) File “/venv/local/lib/python3.6/site-packages/sqlalchemy/sql/compiler.py”, line 2973, in visit_null “type on this Column?” % type) sqlalchemy.exc.CompileError: Can’t generate DDL for NullType(); did you forget to specify a type on this Column?

Images.

Error banner list of tables after test testing connection success importing table

[knz]

The error messages suggest that one of the table’s columns use a SQL data type not supported by either the adapter or by CockroachDB.

It would help to share the data model with us (you can anonymize the table / column names, but please share the types with us).

Thank you

[zacps]

Can confirm this fixes the issue for me.