null cells shift columns when `maxRowsInMemory` enabled

When I specify .option("maxRowsInMemory", 100) then the empty cells are “skipped”, for example:

A	B	C
value	b0	c0
null	b1	c1

without maxRowsInMemory second row would be read as null | b1 | c1. But when I specify the option I get null | c1 | null.

Even when I just save the file again on mac the problem goes away, and I can’t upload the original file.

I tried to set on/off treatEmptyValuesAsNulls, and tried different maxRowsInMemory values, but the problem persists.

Environment:

• Spark 2.2.0
• Scala 2.11.8
• Spark-Excel 0.11.0
• environment: local[*]
• os macOS Sierra 10.12.6

[PanAeon]

No, I had to build my own version to overcome this problem. I think sbt can build a dependency from github, so we don’t have to wait for the release, but it sounds like too much hassle for me.

On Thu, 28 Mar 2019, 12:09 paulpaul1076, [e-mail] wrote:

@PanAeon https://github.com/PanAeon do you know when they will release it though? We are using this library in production and it’s frustrating that this doesn’t work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/crealytics/spark-excel/issues/104#issuecomment-477568690, or mute the thread https://github.com/notifications/unsubscribe-auth/AAp3_LT0BZsQQi9h7dM_yToagGRjg7CTks5vbLESgaJpZM4Z7zCv .

[ajaydubey777]

Hi @PanAeon, I am getting error while using crealytics to read excel file of size 30 MB, were you abel to fix that issue and give some pointer