Error with postgres columns of type "TIMESTAMP WITH TIME ZONE"

• I have checked the superset logs for python stacktraces and included it here as text if any
• I have reproduced the issue with at least the latest released version of superset
• I have checked the issue tracker for the same issue and I haven’t found one similar

Hey! I’m using SQL Lab with a postgres backend. I can’t select from some tables, specifically when the column has a type of TIMESTAMP WITH TIME ZONE. An example of the contents of such a cell is 2014-11-24 11:30:02.140381+00

I have other tables with the column of type TIMESTAMP WITHOUT TIME ZONE, and they work fine. Ex: 2010-01-28T16:59:52.900963

Superset version

0.15.1

Expected results

Query should run and return results.

Actual results

I get flashed a 1 second error message saying “Datatype not understood” which then disappears, and then get shown the loading-spinner.

Stack trace:

2017-01-09 17:09:52,437:INFO:root:Running query:
SELECT product_id,
       base_product_id,
       updated
FROM public.product_id_mapping LIMIT 100
2017-01-09 17:09:52,513:ERROR:root:data type not understood
Traceback (most recent call last):
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/views.py", line 2500, in sql_json
    data = sql_lab.get_sql_results(query_id, return_results=True)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/celery/local.py", line 188, in __call__
    return self._get_current_object()(*a, **kw)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/celery/app/task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/sql_lab.py", line 171, in get_sql_results
    payload['columns'] = cdf.columns_dict if cdf else []
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/dataframe.py", line 54, in columns_dict
    agg = agg_func(self.__df.dtypes[col], col)
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/superset/dataframe.py", line 110, in agg_func
    if np.issubdtype(dtype, np.number):
  File "/Users/user/src/superset/venv/lib/python2.7/site-packages/numpy/core/numerictypes.py", line 755, in issubdtype
    return issubclass(dtype(arg1).type, arg2)
TypeError: data type not understood

Steps to reproduce

Run a SELECT query against a postgresql db table that has a column of type TIMEZONE WITH TIME STAMP

[tfellison]

After poking at this a bit more, it appears the issue is not entirely resolved by updating Pandas. The behavior I’ve observed is as follows:

Prior to updating, an exception would be thrown on any selection from a table containing a timestamp with time zone column, regardless of whether or not that column was selected. After updating, it’s possible to select from such a table, as long as the results themselves do not contain any timestamp with time zone type values. It does appear to be possible to select a timestamp with time zone column as long as it’s you cast to a timestamp without time zone. For example…

SELECT (timevalue_column_name at time zone 'UTC')::timestamp without time zone AS "Timestamp"
FROM table_name;

Applying some function that returns a different type also seems to work fine - converting to date for example:

SELECT date(timevalue_column_name at time zone 'UTC') AS "Date"
FROM table_name;

[angelagilhotra]

I’m currently facing the same error, and this is what my requirements.txt looks like:

aenum==2.1.2
aiohttp==3.7.4.post0
alembic==1.6.5
amqp==2.6.1
aniso8601==9.0.1
apache-superset==1.2.0
apispec==3.3.2
appnope==0.1.2
argh==0.26.2
asgiref==3.3.1
asn1crypto==1.4.0
async-generator==1.10
async-timeout==3.0.1
atomicwrites==1.4.0
attrs==21.2.0
Authlib==0.10
azure-common==1.1.27
azure-core==1.16.0
azure-storage-blob==12.8.1
Babel==2.9.1
backcall==0.2.0
backoff==1.11.1
bcrypt==3.2.0
billiard==3.6.4.0
bleach==3.3.1
blinker==1.4
boto3==1.18.1
botocore==1.21.1
Brotli==1.0.9
cachelib==0.1.1
celery==4.4.7
certifi==2021.5.30
cffi==1.14.6
chardet==4.0.0
charset-normalizer==2.0.3
click==7.1.2
click-default-group==1.2.2
colorama==0.4.4
contextlib2==21.6.0
convertdate==2.3.2
cron-descriptor==1.2.24
croniter==1.0.15
cryptography==3.4.7
decorator==4.4.2
defusedxml==0.7.1
Django==3.1.4
dnspython==2.1.0
email-validator==1.1.3
fasteners==0.15
Flask==1.1.4
Flask-AppBuilder==3.3.1
Flask-Babel==1.0.0
Flask-BabelEx==0.9.4
Flask-Caching==1.10.1
Flask-Compress==1.10.1
Flask-Cors==3.0.10
Flask-Executor==0.9.4
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-Mail==0.9.1
Flask-Migrate==3.0.1
Flask-OpenID==1.2.5
Flask-Principal==0.4.0
Flask-RESTful==0.3.9
Flask-SQLAlchemy==2.5.1
flask-talisman==0.8.1
Flask-WTF==0.14.3
flatten-dict==0.1.0
geographiclib==1.52
geopy==2.2.0
graphlib-backport==1.0.3
greenlet==1.1.0
gunicorn==20.0.4
holidays==0.10.3
humanize==3.10.0
idna==2.10
ipython==7.25.0
ipython-genutils==0.2.0
isodate==0.6.0
itsdangerous==1.1.0
jedi==0.18.0
Jinja2==2.11.3
jmespath==0.10.0
jsonschema==3.2.0
kombu==4.6.11
korean-lunar-calendar==0.2.1
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==2.0.1
marshmallow==3.12.2
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.23.1
matplotlib-inline==0.1.2
meltano==1.77.0
meltano-flask-security==0.1.0
monotonic==1.6
msgpack==1.0.2
msrest==0.6.21
multidict==5.1.0
networkx==2.5.1
numpy==1.21.0
oauthlib==3.1.1
oscrypto==1.2.1
packaging==21.0
pandas==1.2.5
parsedatetime==2.6
parso==0.8.2
passlib==1.7.4
pathlib2==2.3.6
pathtools==0.1.2
pexpect==4.8.0
pgsanity==0.2.9
pickleshare==0.7.5
polyline==1.4.0
prison==0.1.3
prompt-toolkit==3.0.19
psutil==5.8.0
psycopg2==2.9.1
psycopg2-binary==2.9.1
ptyprocess==0.7.0
py==1.10.0
pyarrow==3.0.0
pycparser==2.20
pycryptodomex==3.10.1
Pygments==2.9.0
pyhocon==0.3.58
pyhumps==1.2.2
PyJWT==1.7.1
PyMeeus==0.5.11
pyOpenSSL==20.0.1
pyparsing==2.4.7
PyPika==0.25.2
pyrsistent==0.18.0
python-dateutil==2.8.2
python-dotenv==0.14.0
python-editor==1.0.4
python-geohash==0.8.5
python-gitlab==1.15.0
python3-openid==3.2.0
pytz==2020.5
PyYAML==5.4.1
redis==3.5.3
requests==2.26.0
requests-oauthlib==1.3.0
retry==0.9.2
s3transfer==0.5.0
selenium==3.141.0
simplejson==3.17.3
six==1.16.0
slackclient==2.5.0
smtpapi==0.4.7
snowflake-connector-python==2.4.6
snowflake-sqlalchemy==1.2.4
speaklater==1.3
SQLAlchemy==1.3.24
SQLAlchemy-Utils==0.36.8
sqlparse==0.3.0
traitlets==5.0.5
typing-extensions==3.10.0.0
urllib3==1.26.6
vine==1.3.0
watchdog==0.9.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==0.16.1
WTForms==2.3.3
WTForms-JSON==0.3.3
yarl==1.6.3

what are the steps I should take to fix the issues? (really new to this tech stack, hence a noob question)