question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Not table to get execution plan

See original GitHub issue

Steps to reproduce the behavior:

1.explain insert overwrite table xxx_hudi PARTITION(pt=‘1’). (all the tables involved are hudi tables) select a.id,b.name from emp a, dep b where a.dep_id=b.id

Environment Description

  • Hudi version : 0.9

  • Spark version : 3.0.1

  • Hive version : 3.2

  • Hadoop version : 3.2.2

  • Storage (HDFS/S3/GCS…) : s3

  • Running on Docker? (yes/no) :no

I only get the below information: == Physical Plan == Execute InsertIntoHoodieTableCommand ± InsertIntoHoodieTableCommand Relation[_hoodie_commit_time#317,_hoodie_commit_seqno#318,_hoodie_record_key#319,_hoodie_partition_path#320,_hoodie_file_name#321,one_id#322,phone#323,email#324,activity_type#325,completiondate#326,duedate#327,vehicletype#328,pt#329] parquet, Project [null AS _hoodie_commit_time#312, null AS _hoodie_commit_seqno#313, null AS _hoodie_record_key#314, null AS _hoodie_partition_path#315, null AS _hoodie_file_name#316, one_id#308, AS #311, ids#309, type#16 AS activity_type#0, completiondate#35, duedate#21, vehicletype#33], Map(pt -> Some(1)), true

If I change the target insert table from hudi table to hive table Than the execution plan looks fine: Execute InsertIntoHiveTable dwd_xxx.xxxxxx, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Map(pt -> Some(1)), true, false, [one_id, phone, email, activity_type, completiondate, duedate, vehicletype] ± *(13) Project [ansi_cast(one_id#638 as int) AS one_id#952, AS phone#953, ansi_cast(ids#639 as string) AS email#954, ansi_cast(type#346 as string) AS activity_type#955, ansi_cast(completiondate#365 as timestamp) AS completiondate#956, ansi_cast(duedate#351 as timestamp) AS duedate#957, ansi_cast(vehicletype#363 as string) AS vehicletype#958] ± *(13) SortMergeJoin [email#539], [ids#639], Inner :- *(10) Sort [email#539 ASC NULLS FIRST], false, 0 : ± Exchange hashpartitioning(email#539, 200), true, [id=#162] : ± *(9) Project [completiondate#365, duedate#351, vehicletype#363, type#346, email#539] : ± *(9) SortMergeJoin [accountid#419], [id#531], Inner : :- *(6) Sort [accountid#419 ASC NULLS FIRST], false, 0 : : ± Exchange hashpartitioning(accountid#419, 200), true, [id=#144] : : ± *(5) Project [completiondate#365, duedate#351, vehicletype#363, type#346, accountid#419] : : ± *(5) SortMergeJoin [relatedobjectid#348], [id#412], Inner : : :- *(2) Sort [relatedobjectid#348 ASC NULLS FIRST], false, 0 : : : ± Exchange hashpartitioning(relatedobjectid#348, 200), true, [id=#126] : : : ± *(1) Project [completiondate#365, relatedobjectid#348, duedate#351, vehicletype#363, type#346] : : : ± *(1) Filter type#346 IN (3,7) : : : ± *(1) ColumnarToRow : : :

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
njalancommented, Feb 24, 2022

@XuQianJin-Stars Thanks for your help. I am able to get execution plan for inserting hudi table and execution plan looks fine to me based the jar you shared with me today.

1reaction
YannByroncommented, Feb 24, 2022

@njalan It’s just because InsertIntoHoodieTableCommandextends RunnableCommand and doesm’t override the children method. So it have no children should be shown. @XuQianJin-Stars you can try to override children and return the query in InsertIntoHoodieTableCommand. maybe can show more details.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to obtain SQL Execution Plans using different methods
This article will show the different methods to display SQL execution plans with different methods.
Read more >
SQL SERVER - Find a Table in Execution Plan
Of course yes, we can easily find a table in the execution plan with the Find Node feature of SSMS (SQL Server Management...
Read more >
how to get a better execution plan for empty sets?
I tried creating and joining on empty tables, or top(@somevar) depending on @var 's value - but didn't help. UNION ALL won't ...
Read more >
How do I display and read the execution plans for a SQL ...
EXPLAIN PLAN command - This displays an execution plan for a SQL statement without actually executing the statement.
Read more >
Execution plan over query with temp tables - Stack Overflow
SET SHOWPLAN_TEXT ON (or clicking on Display Estimated Execution Plan is SSMS) creates a plan instead of executing the SQL.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found