question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Bug] SelectNode.cardinality < 0 cause query exception "Unexpected exception: null"

See original GitHub issue

Describe the bug When using select condition like -1=-1 may cause errCode = 2, detailMessage = Unexpected exception: null. This is due toSelectNode.cardinality computing error.

https://github.com/apache/incubator-doris/blob/81aa1c599def3c3f0d9cd80469fd06393ea764e7/fe/src/main/java/org/apache/doris/planner/SelectNode.java#L63-L73

This cardinality may be less than 0.

To Reproduce Steps to reproduce the behavior:

  1. Create a table
CREATE TABLE `table1` (
  `city_id` bigint(20) NULL,
  `tag_id` bigint(20) NULL,
  `dt` bigint(20) NULL,
  `num` bigint(20) NULL
) ENGINE=OLAP
DUPLICATE KEY(`city_id`, `tag_id`, `dt`)
COMMENT "OLAP"
PARTITION BY RANGE(`dt`)
(PARTITION p20200102 VALUES [("20200101"), ("20200102")),
PARTITION p20200103 VALUES [("20200102"), ("20200103")))
DISTRIBUTED BY HASH(`city_id`) BUCKETS 5
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);
  1. Insert some data
INSERT INTO table1 VALUES (1, 2, 20200102, 3);

You may need to wait for the cardinality to be > 0 (I still dont know how the cardinality works).

  1. Launch a query
SELECT rank, tag_id
FROM (
    SELECT row_number() OVER (ORDER BY num DESC) AS rank, tag_id
    FROM table1
    ) t
WHERE -1=-1 or tag_id in (-1)
LIMIT 10
;

When the table is not empty (maybe cardinality means row number?), cardinality will be > 0, computeSelectivity() < 0, then the error occur.

I tried to remove some elements and this is the minimum sql to reproduce the bug.

Could someone explain what doe cardinality and selectivity means? Is Preconditions.checkState(cardinality >= 0); necessary?

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
ccofflinecommented, Sep 10, 2020

-1=-1 or other True expr will turn into a BoolLiteral which selectivity = -1. tag_id in (-1) will turn into a InPredicate which selectivity = 0.1. So in their father node CompoundPredicate, selectivity will be set to -1.

https://github.com/apache/incubator-doris/blob/81aa1c599def3c3f0d9cd80469fd06393ea764e7/fe/src/main/java/org/apache/doris/analysis/CompoundPredicate.java#L106-L115

If remove any of these expresstions, SelectNode.computeStats will not be executed.

0reactions
liutang123commented, Sep 25, 2020

Give your PR linked this issue please.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[Bug] SelectNode.cardinality < 0 cause query exception ...
Describe the bug When using select condition like -1=-1 may cause errCode = 2, detailMessage = Unexpected exception: null.
Read more >
[GitHub] [incubator-doris] ccoffline commented on issue #4581: [Bug ...
[GitHub] [incubator-doris] ccoffline commented on issue #4581: [Bug] SelectNode.cardinality < 0 cause query exception "Unexpected exception: null".
Read more >
System Query Exception: Unexpected Token
I've am receiving an error from a process I created that was working fine earlier today. The error is: Error element myRule_32_A1 ( ......
Read more >
Fixed Issues in Apache Impala | 5.x - Cloudera Documentation
IMPALA-8058 - Fixed cardinality estimates for HBase queries, ... IMPALA-6451 - Fixed the AuthorizationException in CTAS for Kudu tables.
Read more >
Reference Material Red Hat JBoss Data Virtualization 6.2
A null literal used in the SELECT clause of a query with no implied context ... A null non-warning exception will still cause...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found