[SUPPORT] Spark SQL to query metadata such as hudi table snapshot
See original GitHub issueSupport Spark SQL to query metadata such as hudi table snapshot。Example:
#Iceberg sql
SELECT * FROM bigdata.ice_arctic_demo.snapshots;
SELECT * FROM bigdata.ice_arctic_demo.history;
SELECT * FROM bigdata.ice_arctic_demo.files;
SELECT * FROM bigdata.ice_arctic_demo.manifests;
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Querying Data - Apache Hudi
Once the Hudi tables have been registered to the Hive metastore, they can be queried using the Spark-Hive integration. By default, Spark SQL...
Read more >Work with a Hudi dataset - Amazon EMR - AWS Documentation
Hudi supports inserting, updating, and deleting data in Hudi datasets through Spark. For more information, see Writing Hudi tables in Apache Hudi ......
Read more >Table file formats - reading path: Apache Hudi
Copy-On-Write table with snapshot or read-optimized query, Merge-On-Read table with read-optimized query. Here Apache Hudi creates a ...
Read more >[GitHub] [hudi] codejoyan commented on issue #3581: [SUPPORT ...
hudi.apache.org For queries about this service, please contact ... DataSourceReadOptions scala> spark.sql("SET hoodie.metadata.enable=true") res0: ...
Read more >Building a Large-scale Transactional Data Lake at Uber Using ...
To this end, Hudi supports multiple query engine integrations such as Presto, Apache Hive, Apache Spark, and Apache Impala. Figure 1.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@melin @zhangyue19921010 I will use the call procedure command to obtain meta information through SparkSQL. If Directly abstract it into a table to query, and I will evaluate the change scope and workload.
thanks!