Not table to get execution plan
See original GitHub issueSteps to reproduce the behavior:
1.explain insert overwrite table xxx_hudi PARTITION(pt=‘1’). (all the tables involved are hudi tables) select a.id,b.name from emp a, dep b where a.dep_id=b.id
Environment Description
-
Hudi version : 0.9
-
Spark version : 3.0.1
-
Hive version : 3.2
-
Hadoop version : 3.2.2
-
Storage (HDFS/S3/GCS…) : s3
-
Running on Docker? (yes/no) :no
I only get the below information: == Physical Plan == Execute InsertIntoHoodieTableCommand ± InsertIntoHoodieTableCommand Relation[_hoodie_commit_time#317,_hoodie_commit_seqno#318,_hoodie_record_key#319,_hoodie_partition_path#320,_hoodie_file_name#321,one_id#322,phone#323,email#324,activity_type#325,completiondate#326,duedate#327,vehicletype#328,pt#329] parquet, Project [null AS _hoodie_commit_time#312, null AS _hoodie_commit_seqno#313, null AS _hoodie_record_key#314, null AS _hoodie_partition_path#315, null AS _hoodie_file_name#316, one_id#308, AS #311, ids#309, type#16 AS activity_type#0, completiondate#35, duedate#21, vehicletype#33], Map(pt -> Some(1)), true
If I change the target insert table from hudi table to hive table Than the execution plan looks fine:
Execute InsertIntoHiveTable dwd_xxx
.xxxxxx
, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Map(pt -> Some(1)), true, false, [one_id, phone, email, activity_type, completiondate, duedate, vehicletype]
± *(13) Project [ansi_cast(one_id#638 as int) AS one_id#952, AS phone#953, ansi_cast(ids#639 as string) AS email#954, ansi_cast(type#346 as string) AS activity_type#955, ansi_cast(completiondate#365 as timestamp) AS completiondate#956, ansi_cast(duedate#351 as timestamp) AS duedate#957, ansi_cast(vehicletype#363 as string) AS vehicletype#958]
± *(13) SortMergeJoin [email#539], [ids#639], Inner
:- *(10) Sort [email#539 ASC NULLS FIRST], false, 0
: ± Exchange hashpartitioning(email#539, 200), true, [id=#162]
: ± *(9) Project [completiondate#365, duedate#351, vehicletype#363, type#346, email#539]
: ± *(9) SortMergeJoin [accountid#419], [id#531], Inner
: :- *(6) Sort [accountid#419 ASC NULLS FIRST], false, 0
: : ± Exchange hashpartitioning(accountid#419, 200), true, [id=#144]
: : ± *(5) Project [completiondate#365, duedate#351, vehicletype#363, type#346, accountid#419]
: : ± *(5) SortMergeJoin [relatedobjectid#348], [id#412], Inner
: : :- *(2) Sort [relatedobjectid#348 ASC NULLS FIRST], false, 0
: : : ± Exchange hashpartitioning(relatedobjectid#348, 200), true, [id=#126]
: : : ± *(1) Project [completiondate#365, relatedobjectid#348, duedate#351, vehicletype#363, type#346]
: : : ± *(1) Filter type#346 IN (3,7)
: : : ± *(1) ColumnarToRow
: : :
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (6 by maintainers)
Top GitHub Comments
@XuQianJin-Stars Thanks for your help. I am able to get execution plan for inserting hudi table and execution plan looks fine to me based the jar you shared with me today.
@njalan It’s just because
InsertIntoHoodieTableCommand
extendsRunnableCommand
and doesm’t override thechildren
method. So it have no children should be shown. @XuQianJin-Stars you can try to overridechildren
and return thequery
inInsertIntoHoodieTableCommand
. maybe can show more details.