question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Time conversion issue while converting JSON to Parquet.

See original GitHub issue

I am trying to convert JSON file to Parquet in Azure function using the below method:

string jsonFilePath = @"F:\contactdata.json";
string json = File.ReadAllText(jsonFilePath);
using (var reader = ChoJSONReader.LoadText(json))
{
     using (var writer = new ChoParquetWriter(F:\jsontoparquet.parquet))
       {
           writer.Write(reader);
        }
}

But I’m getting exception while executing the above code, the error is:

System.Private.CoreLib: The UTC Offset of the local dateTime parameter does not match the offset argument. (Parameter ‘offset’).

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
Cinchoocommented, Jun 2, 2021

I couldn’t reproduce it locally, But did quick search on internet, found a fix. Applied fix, pushed new package v1.0.1.9. Try and let me know. Hope the fix solve the issue.

0reactions
Hasan-1999commented, Jun 3, 2021

I couldn’t reproduce it locally, But did quick search on internet, found a fix. Applied fix, pushed new package v1.0.1.9. Try and let me know. Hope the fix solve the issue.

Thank You for your response. Time Conversion issue is fixed now. I can convert JSON file to Parquet file now.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to convert a JSON result to Parquet in python?
Here's how to convert a JSON file to Apache Parquet format, using Pandas in Python. This is an easy method with a well-known...
Read more >
Unable to convert datetime field from json to parquet #1066
we want to convert json message to parquet and we have a protobuf schema like. syntax = "proto3";. import "google/protobuf/timestamp.proto";.
Read more >
Conversion of JSON to parquet format using Apache ...
To convert JSON data files to Parquet, you need some in-memory representation. Parquet doesn't have its own set of Java… ... Once the...
Read more >
Resolve wrong Date Conversion in Spark when importing ...
Issue : Parquet file exported from Snowflake that contains date column results in incorrect date value when imported into Spark 2.4 or earlier....
Read more >
How to best convert JSON to parquet files?
Dump the json into an obj, parse the obj, load into dataframe, convert the dataframe to parquet, and send to blob storage (S3)....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found