Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Receiving NULL values when querying array column types in hive
See original GitHub issueI have a table in hive that has a column of array(varchar) type.
Querying this column in v0.164 and earlier returns correct values.
Querying the column in v0.165 and above returns NULL values.
The hive config file is:
connector.name=hive-hadoop2
hive.metastore.uri=thrift://<hostname>:9083
hive.config.resources=/etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml
hive.parquet.use-column-names=true
hive.parquet-optimized-reader.enabled=true
hive.parquet-predicate-pushdown.enabled=true
hive.storage-format=PARQUET
hive.compression-codec=SNAPPY
If I disablehive.parquet-optimized-reader setting, then it returns correct values in v0.165 and above, however, I then encounter an different issue regarding reading DECIMAL column types.
Issue Analytics
- State:
- Created 6 years ago
- Comments:13 (3 by maintainers)
Top Results From Across the Web
Handling NULL values in Hive - Stack Overflow
Firstly — I don't think column1 is not NULL or column1 <> '' makes very much sense. Maybe you meant to write column1...
Read more >Why Hive Table is loading with NULL values?
You are getting NULL values loaded to the Hive table because your data is in comma-separated whereas Hive default separator is ^A hence...
Read more >How to insert NULL value into Hive complex columns... - 203625
We have a Hive table with some columns being arrays and structs. When inserting a row into the table, if we do -...
Read more >Testing a Hive array for IS NULL says not null
I just investigated the same issue: In my case it was due to Hive CBO (Cost based optimizer) erroneously replacing the IS NULL...
Read more >Querying arrays with complex types and nested structures
Changing field names in arrays using CAST ... In the example above, you declare name as a VARCHAR because this is its type...
Read more >
Top Related Medium Post
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@nezihyigitbasi using the newer version fixed the issue. Thanks!
@sj860908 Did you try Presto 0.203+? The optimized Parquet reader has several fixes for proper complex type handling.