[SUPPORT]hudi sync hive failed no data in hive help!!! please!!!
See original GitHub issueTips before filing an issue
-
Have you gone through our FAQs?
-
yes
-
Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
-
If you have triaged this as a bug, then file an issue directly.
Describe the problem you faced
A clear and concise description of the problem.
To Reproduce
Steps to reproduce the behavior:
CREATE TABLE t23(
uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED, name VARCHAR(10) ) WITH ( ‘connector’ = ‘hudi’, ‘path’ = ‘hdfs://localhost:9000/user/data/hudi/t23’, ‘table.type’ = ‘MERGE_ON_READ’, ‘hive_sync.enable’= ‘true’, ‘hive_sync.mode’= ‘hms’, ‘hive_sync.metastore.uris’=‘thrift://localhost:9083’, ‘hive_sync. jdbc_url’=‘jdbc:hive2://localhost:10000’, ‘hive_sync.table’= ‘t23’, ‘hive sync.db’= ‘default’, ‘hive_sync.username’= ‘hive’, ‘hive_sync.password’= ‘hive’, ‘hive sync.support_timestamp’=‘true’ );
INSERT INTO t23 SELECT * FROM t3_kafka_sour; 3. in hive SELECT * FROM t23_ro;
Expected behavior
t23_ro have data
Environment Description
-
Hudi version :0.11.0
-
Spark version :
-
Hive version :2.3.9
-
Hadoop version :2.7.3
-
Storage (HDFS/S3/GCS…) :hdfs
-
Running on Docker? (yes/no) :no
Additional context
0: jdbc:hive2://localhost:10000> SELECT * FROM t23_ro;
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
| t23_ro._hoodie_commit_time | t23_ro._hoodie_commit_seqno | t23_ro._hoodie_record_key | t23_ro._hoodie_partition_path | t23_ro._hoodie_file_name | t23_ro.uuid | t23_ro.name |
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
No rows selected (0.212 seconds)
0: jdbc:hive2://localhost:10000>
in hudi
Table program finished. Page: Last of 1 Updated: 19:08:56.854
uuid name
1 qq
100 201
2 38
101 202
3 78
102 203
4 88
103 204
5 90
104 205
6 100
105 206
7 201
106 207
8 201
Stacktrace
there is no error but there is no data in hive
Issue Analytics
- State:
- Created a year ago
- Comments:9 (5 by maintainers)
Top GitHub Comments
I solved this by open compaction
I think we need that, especially for hive reading MOR table with pure logs.