Why query `system.jdbc.columns` so slow.

env

• presto 0.210
• jdk1.8
• dbeaver 5.2.4
• DataGrip 2018.2.4

dbeaver

When query from a ‘new’ table - hasn’t been queried in current connection - say select * from t_sales_bill or select bill_id from t_sales_bill it will automatically run the sql below to query all columns in t_sales_bill table AND SO SLOW(cost 36s)!! It happen whenever you query from a ‘new’ table and so wasting time!

SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE,
  TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX,
  NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB,
  CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE,
  SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE,
  SOURCE_DATA_TYPE, IS_AUTOINCREMENT, IS_GENERATEDCOLUMN
FROM system.jdbc.columns
WHERE TABLE_CAT = 'platform_data' AND TABLE_SCHEM LIKE 'platform_data' ESCAPE '\' AND TABLE_NAME LIKE 't_sales_bill' ESCAPE '\' AND COLUMN_NAME LIKE '%' ESCAPE '\'
ORDER BY TABLE_CAT, TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION

DataGrip

DataGrip has the same problem like dbeaver but not so bad but BAD.

DataGrip will query the whole metadata in system.jdbc when connection started.

So, it’s like ‘once for all’ not when you need it. BUT it took 55s to query system.jdbc.columns!

SO

• Why query system.jdbc.columns so slow?
• Any better database tools for presto?

[kokosing]

It looks like your query is slow because dbeaver has a bug. This is very typical bug.

Please take a look at the query:

SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE,
  TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX,
  NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB,
  CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE,
  SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE,
  SOURCE_DATA_TYPE, IS_AUTOINCREMENT, IS_GENERATEDCOLUMN
FROM system.jdbc.columns
WHERE TABLE_CAT = 'platform_data' AND TABLE_SCHEM LIKE 'platform_data' ESCAPE '\' AND TABLE_NAME LIKE 't_sales_bill' ESCAPE '\' AND COLUMN_NAME LIKE '%' ESCAPE '\'
ORDER BY TABLE_CAT, TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION

And specifically predicates: TABLE_SCHEM LIKE 'platform_data' ESCAPE '\' and TABLE_NAME LIKE 't_sales_bill' ESCAPE '\'. Notice that your object names contains _, in pattern for LIKE operator _ means any character and in your case it should be escaped. These predicates should look like: TABLE_SCHEM LIKE 'platform\_data' ESCAPE '\' and TABLE_NAME LIKE 't\_sales\_bill' ESCAPE '\'.

Can you please rerun your query but with properly escaped LIKE patterns literals? That would be:

SELECT TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE,
  TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX,
  NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB,
  CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE,
  SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE,
  SOURCE_DATA_TYPE, IS_AUTOINCREMENT, IS_GENERATEDCOLUMN
FROM system.jdbc.columns
WHERE TABLE_CAT = 'platform\_data' AND TABLE_SCHEM LIKE 'platform\_data' ESCAPE '\' AND TABLE_NAME LIKE 't\_sales\_bill' ESCAPE '\' AND COLUMN_NAME LIKE '%' ESCAPE '\'
ORDER BY TABLE_CAT, TABLE_SCHEM, TABLE_NAME, ORDINAL_POSITION

[findepi]

Let me close this in favor of https://github.com/dbeaver/dbeaver/issues/4587