question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Presto Timestamp types are being identified as UNKNOWN datatype

See original GitHub issue

A clear and concise description of what the bug is.

PrestoSQL Version - 341 SuperSet Version - 0.37.2

I have a table in Presto that has 1250 columns. 118 of these columns are timestamp type. When I add this table under Sources -> Tables -> Add a new record, all these timestamps are being identified as UNKNOWN. However, when I manually edit each timestamp column and change UNKNOWN -> TIMESTAMP, it all works fine as expected.

I tried the same in master build docker image. Data -> Datasets -> Add a new record referencing the table in DB. All timestamp fields are being identified as UNKNOWN type.

Adding the DB table in the below screen image

what you expected to happen. timestamp(3) which is the native timestamp of Presto should be identified as timestamp image

what actually happens.

Screenshots

image

If applicable, add screenshots to help explain your problem. Below screenshot is shown from the image built from master: image

How to reproduce the bug

  1. Have a Presto table defined with timestamp field in it.
  2. Add a presto connection in the database section.
  3. Go to Sources -> Tables -> Add a new record
  4. Click on edit record -> COLUMNS -> check the timestamp type you stored in DB.

Environment

(please complete the following information):

Checklist

Make sure these boxes are checked before submitting your issue - thank you!

  • [y] I have checked the superset logs for python stacktraces and included it here as text if there are any.
  • [y] I have reproduced the issue with at least the latest released version of superset.
  • [y] I have checked the issue tracker for the same issue and I haven’t found one similar.

Additional context

Add any other context about the problem here.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
DRavikanthcommented, Oct 2, 2020

So, to whoever is running into this problem, the issue should have happened post-Presto 338 release where the timestamp is a parametric timestamp type (timestamp with different precision).

https://github.com/prestosql/presto/pull/4349

This is what I did to resolve the problem. Updated my database connection and specific extra field with the following:

{
  "metadata_params": {},
  "engine_params": {
    "connect_args": {
      "protocol": "https",
      "session_props": {
          "omit_datetime_type_precision": true
      }
    }
  },
  "metadata_cache_timeout": {},
  "schemas_allowed_for_csv_upload": []
}

Make a note of the “omit_datetime_type_precision”: true property. Refer: https://prestosql.io/docs/current/release/release-338.html

This works fine now after I added the flag. I will leave it to Superset developers if they want to add a fix for it.

0reactions
DRavikanthcommented, Oct 2, 2020

Sounds good.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Presto Timestamp types are being identified as UNKNOWN ...
Have a Presto table defined with timestamp field in it. Add a presto connection in the database section. ... Click on edit record...
Read more >
Presto odbc failing to read timestamp fileds in HIVE
Unfortunately, Simba's ODBC driver makes certain assumptions about how the type names are presented to the client and fails with the new types....
Read more >
Data Types — Presto 0.278.1 Documentation
Presto has a set of built-in data types, described below. Additional types can be provided by plugins. Note. Connectors are not required to...
Read more >
Resolve timestamp exceptions when querying a table in ...
When I query a column of TIMESTAMP data in my Amazon Athena table, ... on a column with the data type that's not...
Read more >
(1070) Unknown Presto data type: timestamp(6)
(1070) Unknown Presto data type: timestamp(6). 5 hours ago. Hi everyone! I have been using an ODBC connector to access my data in...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found