Time conversion issue while converting JSON to Parquet.
See original GitHub issueI am trying to convert JSON file to Parquet in Azure function using the below method:
string jsonFilePath = @"F:\contactdata.json";
string json = File.ReadAllText(jsonFilePath);
using (var reader = ChoJSONReader.LoadText(json))
{
using (var writer = new ChoParquetWriter(F:\jsontoparquet.parquet))
{
writer.Write(reader);
}
}
But I’m getting exception while executing the above code, the error is:
System.Private.CoreLib: The UTC Offset of the local dateTime parameter does not match the offset argument. (Parameter ‘offset’).
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
How to convert a JSON result to Parquet in python?
Here's how to convert a JSON file to Apache Parquet format, using Pandas in Python. This is an easy method with a well-known...
Read more >Unable to convert datetime field from json to parquet #1066
we want to convert json message to parquet and we have a protobuf schema like. syntax = "proto3";. import "google/protobuf/timestamp.proto";.
Read more >Conversion of JSON to parquet format using Apache ...
To convert JSON data files to Parquet, you need some in-memory representation. Parquet doesn't have its own set of Java… ... Once the...
Read more >Resolve wrong Date Conversion in Spark when importing ...
Issue : Parquet file exported from Snowflake that contains date column results in incorrect date value when imported into Spark 2.4 or earlier....
Read more >How to best convert JSON to parquet files?
Dump the json into an obj, parse the obj, load into dataframe, convert the dataframe to parquet, and send to blob storage (S3)....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I couldn’t reproduce it locally, But did quick search on internet, found a fix. Applied fix, pushed new package v1.0.1.9. Try and let me know. Hope the fix solve the issue.
Thank You for your response. Time Conversion issue is fixed now. I can convert JSON file to Parquet file now.