question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Column order is incorrect and nondeterministic when writing to CSV with capital letters

See original GitHub issue

When serializing to a CSV the column order output is incorrect but only when one or more attributes has a capital letter. The output order is also nondeterministic. Two back-to-back runs with the exact same values can produce different output orders.

Code to reproduce bug (Kotlin):

package com.draper.surveillance.common

import com.fasterxml.jackson.annotation.JsonPropertyOrder
import com.fasterxml.jackson.databind.MapperFeature
import com.fasterxml.jackson.databind.ObjectWriter
import com.fasterxml.jackson.dataformat.csv.CsvMapper
import java.io.File


class MyClass(
   val zProp1: Int,
   val Prop2: Int,
   val Prop3: Int)

val csvMapper = CsvMapper().apply {
   disable(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY)
}
val bootstrapSchema = csvMapper.schemaFor(MyClass::class.java)
   .withUseHeader(true)
val writer: ObjectWriter = csvMapper
   .writerFor(MyClass::class.java)
   .with(bootstrapSchema)

val sequenceWriter = writer.writeValues(File("test.csv"))
object Test {
   val l = listOf(
      MyClass(1,2,3),
      MyClass(4,5,6)
   )

   @JvmStatic
   fun main(args: Array<String>) {
      sequenceWriter.writeAll(l)
   }

}

Running the main method multiple times will result outputs with different orders. If the property names in MyClass are changed to all lowercase then the issue disappears. This issue has been confirmed to happen in Java as well, version 2.13.0. See this.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
cowtowncodercommented, Nov 26, 2021

Right: earlier JDKs tended to return Fields and Methods in order but even that is not guaranteed and apparently has lately been changing. In addition, combination ordering across Fields/Methods is not defined and use of inheritance can basically make it impossible to find out full declaration order anyway. In hindsight it would probably have made for Jackson to default to alphabetic ordering from version 1.0. But at the time it was not as obvious as later on; and wasn’t as much an issue with JSON anyway as ordering should not have semantic meaning. This is definitely different with positional formats like CSV.

1reaction
JacobGH111commented, Nov 25, 2021

Thank you, this is tremendously helpful and now I understand why this is not a bug but just a limitation of the JVM.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Kotlin Jackson serilization wrong column order and ...
When serializing to a CSV the column order output is incorrect but only when one or more attributes has a capital letter.
Read more >
Output to .csv Files sometimes Switches/incorrect Column Order
Customer has several reports that they output with the data type .csv. ... the the column order in the .csv file remains consistent....
Read more >
opencsv –
Opencsv is an easy-to-use CSV (comma-separated values) parser library for Java. It was developed because all the CSV parsers at the time didn't...
Read more >
Migration Guide: SQL, Datasets and DataFrame - Apache Spark
Since Spark 3.3, nulls are written as empty strings in CSV data source by ... In Spark 3.1 and earlier, it used to...
Read more >
Working with DataFrames in Snowpark Scala
In order to retrieve the data into the DataFrame, you must invoke a method that ... For example, the following table name does...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found