Add sample high/low watermark data to quick start
See original GitHub issueWe use
SELECT From_unixtime(A0.create_time) as create_time,
C0.NAME as schema_name,
B0.tbl_name as table_name,
{func}(A0.part_name) as part_name,
{watermark} as part_type
FROM PARTITIONS A0
LEFT OUTER JOIN TBLS B0
ON A0.tbl_id = B0.tbl_id
LEFT OUTER JOIN DBS C0
ON B0.db_id = C0.db_id
WHERE C0.NAME IN {schemas}
AND B0.tbl_type IN ( 'EXTERNAL_TABLE', 'MANAGED_TABLE' )
AND A0.PART_NAME NOT LIKE '%%__HIVE_DEFAULT_PARTITION__%%'
GROUP BY C0.NAME, B0.tbl_name
ORDER by create_time desc
as the high metastore query to get the high watermark and low watermark for a hive table. And we have already included the datamodel in https://github.com/lyft/amundsendatabuilder/blob/master/databuilder/models/hive_watermark.py .
We should be able to provide a sample data with csv format to quick start. UI has already had support with watermark data.
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (5 by maintainers)
Top Results From Across the Web
Structured Streaming Programming Guide - Apache Spark
First, let's start with a simple example of a Structured Streaming query - a streaming word count. Quick Example. Let's say you want...
Read more >Add watermarks and background objects in Pages on Mac
In Pages on your Mac, add section layout objects, such as watermarks and logos, that appear in the same place on the background...
Read more >Incremental or High Water Mark data Loading
In this article, we'll examine one way that we can easily perform incremental loads using data staging connectors inside Matillion ETL. The ...
Read more >Charting in the Active Trader Pro Platforms
Place your cursor on the start of the time range you would like to analyze, and simply click and drag your cursor to...
Read more >JasperReports - Quick Guide - Tutorialspoint
JasperCompileManager − Used to compile a JRXML report template. net.sf.jasperreports.engine.JasperFillManager − Used to fill a report with data from the data ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@Mikhail-Ivanov and @feng-tao anything else needed to close this with https://github.com/lyft/amundsendatabuilder/pull/145 merged?
@Mikhail-Ivanov all yours 😃