Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Investigate if Hive connector should add stats properties

See original GitHub issue

Partitions (or unpartitioned tables) written by Hive seem to have the following properties (tested with Hive 1.2.1):

COLUMN_STATS_ACCURATE = true
numFiles
numRows
rawDataSize
totalSize

This looks like a good starting point: https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/common/StatsSetupConst.java

Issue Analytics

State:
Created 7 years ago
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

arhimondrcommented, Apr 19, 2018

Basic statistics: https://github.com/prestodb/presto/pull/10456

0reactions

arhimondrcommented, Jun 11, 2018

Yes

Top Results From Across the Web

Hadoop Hive - Configuration Properties

Typically set to a prime close to the number of available hosts. Ignored when mapred.job.tracker is "local". Hadoop set this to 1 by...

Statistics in Hive - Apache Software Foundation

Queries can fail to collect stats completely accurately. There is a setting hive.stats.reliable that fails queries if the stats can't be reliably collected....

Hive connector — Trino 403 Documentation

The Hive connector allows querying data stored in an Apache Hive data warehouse. Hive is a combination of three components: Data files in...

2.04 - Hive Connector and Link Properties - Teradata QueryGrid

When you create links and associated properties in the QueryGrid portlet, you are creating Configuration Name Value Pairs (NVP).

Hive Connector — Presto 0.278 Documentation

We also recommend reducing the configuration files to have the minimum set of required properties, as additional properties may cause problems. The ...