question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Semantic inconsistency use coral convert hive query to presto

See original GitHub issue

You can use the following test case to reproduce the issue.

  public static final TestTable TABLE_FIVE = new TestTable("dwd_ogv_app_play_click_tf_inc_d", ImmutableMap.of("r_type", SqlTypeName.INTEGER,
      "ogv_type_name", SqlTypeName.VARCHAR, "buvid", SqlTypeName.VARCHAR, "log_date", SqlTypeName.VARCHAR));

  @Test
  public void testSelect() {
    String sql = String.format("select t.ogv_type_name, count(1) as user_amt from (SELECT ogv_type_name,\n" +
        "buvid\n" +
        "FROM dwd_ogv_app_play_click_tf_inc_d\n" +
        "WHERE log_date = '20210312'\n" +
        "AND r_type = 1\n" +
        "AND ogv_type_name IS NOT NULL\n" +
        "GROUP BY ogv_type_name, buvid) t GROUP BY\n" +
        "t.ogv_type_name", tableFive);
    String expectedSql = "";
    testConversion(sql, expectedSql);
  }

After convert, the presto query is below:

SELECT "ogv_type_name" AS "OGV_TYPE_NAME", COUNT(*) AS "USER_AMT"
FROM "dwd_ogv_app_play_click_tf_inc_d"
WHERE "log_date" = '20210312' AND "r_type" = 1 AND "ogv_type_name" IS NOT NULL
GROUP BY "ogv_type_name"

The semantics of the two Queries are clearly inconsistent.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
huangxiaopingRDcommented, May 7, 2021

this bug is relate to calcite,the patch solved the problem. https://issues.apache.org/jira/browse/CALCITE-3466

Read more comments on GitHub >

github_iconTop Results From Across the Web

How can I Migrate Hive Queries into Presto
Review the migration guide from Hive. Presto follows the ANSI SQL syntax and semantics, while Hive uses a different SQL syntax dialect.
Read more >
Coral & Transport - Databricks
Coral IR captures query semantics using standard operators. • Supported Transformations. • Hive QL (optionally Spark SQL) to Coral IR.
Read more >
Coral: A SQL translation, analysis, and rewrite engine for ...
After obtaining the Coral IR of a view definition using Coral Hive, Coral Presto performs the conversion through a two-step process:
Read more >
Blog Archive | Ahana
AWS Athena is an interactive query service based on Presto that makes it easy to analyze data in Amazon S3 using standard SQL....
Read more >
Hive ACID and transactional tables' support in Presto - Trino
Several concepts like transactions, WriteIds, deltas, locks, etc. are added in Hive to achieve ACID semantics. To understand the changes done in ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found