question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Invalid parquet type: DECIMAL

See original GitHub issue

When using await parquet.ParquetReader.openFile("so.parquet"); i get the error invalid parquet type: DECIMAL since it is not included in PARQUET_LOGICAL_TYPES. It is listed as one of the logical types here: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md

Any chance it will be added soon?

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:21
  • Comments:7

github_iconTop GitHub Comments

2reactions
ryanvaloriscommented, Mar 18, 2022

There does seem to be a pull request that would fix this issue here - https://github.com/ZJONSSON/parquetjs/pull/44/files

However, this pull request is in a fork of this package

0reactions
ryanvaloriscommented, Mar 18, 2022

as am I

Read more comments on GitHub >

github_iconTop Results From Across the Web

Incorrect decimal data when parquet file is processed with ...
When processing decimal data with ADLS Gen2 connector in Parquet format, the value loaded is incorrect. ... 1) For Solution, enter CR with...
Read more >
Issue while reading a parquet file with different data types like ...
one thing I noticed when I ran the same code with pandas dataframe it worked perfectly means the decimal columns are consider as...
Read more >
Data type issue for decimal fields in parquet files between ...
I am generating parquet files using Data Flow in ADF. The files have some fields with decimal data types.
Read more >
Parquet Decimal Datatype Returning an Error in Impala
We are having an issue reading tables in Impala with datatype Decimal (1,1). The error is: column 'DECIMAL_1_1' has an invalid type length....
Read more >
[#IMPALA-7087] Impala is unable to read Parquet decimal ...
ERROR: File '/hdfs/path/000000_0_x_2' column 'alterd_decimal' has an invalid type length. Expecting: 11 len in file: 8 It would be ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found