question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[SUPPORT]hudi sync hive failed no data in hive help!!! please!!!

See original GitHub issue

Tips before filing an issue

  • Have you gone through our FAQs?

  • yes

  • Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

  • If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced

A clear and concise description of the problem.

To Reproduce

Steps to reproduce the behavior:

CREATE TABLE t23(

uuid VARCHAR(20) PRIMARY KEY NOT ENFORCED, name VARCHAR(10) ) WITH ( ‘connector’ = ‘hudi’, ‘path’ = ‘hdfs://localhost:9000/user/data/hudi/t23’, ‘table.type’ = ‘MERGE_ON_READ’, ‘hive_sync.enable’= ‘true’, ‘hive_sync.mode’= ‘hms’, ‘hive_sync.metastore.uris’=‘thrift://localhost:9083’, ‘hive_sync. jdbc_url’=‘jdbc:hive2://localhost:10000’, ‘hive_sync.table’= ‘t23’, ‘hive sync.db’= ‘default’, ‘hive_sync.username’= ‘hive’, ‘hive_sync.password’= ‘hive’, ‘hive sync.support_timestamp’=‘true’ );

INSERT INTO t23 SELECT * FROM t3_kafka_sour; 3. in hive SELECT * FROM t23_ro;

Expected behavior

t23_ro have data

Environment Description

  • Hudi version :0.11.0

  • Spark version :

  • Hive version :2.3.9

  • Hadoop version :2.7.3

  • Storage (HDFS/S3/GCS…) :hdfs

  • Running on Docker? (yes/no) :no

Additional context

0: jdbc:hive2://localhost:10000> SELECT * FROM t23_ro;
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
| t23_ro._hoodie_commit_time  | t23_ro._hoodie_commit_seqno  | t23_ro._hoodie_record_key  | t23_ro._hoodie_partition_path  | t23_ro._hoodie_file_name  | t23_ro.uuid  | t23_ro.name  |
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
+-----------------------------+------------------------------+----------------------------+--------------------------------+---------------------------+--------------+--------------+
No rows selected (0.212 seconds)
0: jdbc:hive2://localhost:10000> 
in hudi

Table program finished.                     Page: Last of 1                       Updated: 19:08:56.854 

                           uuid                           name
                              1                             qq
                            100                            201
                              2                             38
                            101                            202
                              3                             78
                            102                            203
                              4                             88
                            103                            204
                              5                             90
                            104                            205
                              6                            100
                            105                            206
                              7                            201
                            106                            207
                              8                            201

Stacktrace

there is no error but there is no data in hive

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
ykPrograWorldcommented, Jun 10, 2022

I solved this by open compaction

0reactions
danny0405commented, Jun 14, 2022

I think we need that, especially for hive reading MOR table with pure logs.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Hive table is not reflecting in atlas automatically.
when i run the import-hive.sh file manually tables getting sync with atlas and when i debug more i found that - 193688.
Read more >
Problems and workarounds: Hive - IBM
A Hive create table operation fails with the following DDLTask error: MetaException(message:java.lang.IllegalArgumentException: /user/bigsql/sync is not a ...
Read more >
Query externally partitioned data in Cloud Storage | BigQuery
Hive partitioning support is enabled by setting the appropriate options in the ... are not encoded in the directory path, partition schema detection...
Read more >
Hive Metastore - Apache Hudi
Hive Sync Tool​. Writing data with DataSource writer or HoodieDeltaStreamer supports syncing of the table's latest schema to Hive metastore, such that queries ......
Read more >
Hive command examples for exporting, importing, and ...
The following examples use Hive commands to perform operations such as exporting data to Amazon S3 or HDFS, importing data to DynamoDB, joining...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found