NOT NULL constraint failed for non-nullable Charfields in xlsx

When using the .xslx import, non-nullable CharFields will fail with an exception when their value is left empty. Presumably because the cell has a value of None in that case.

As seen on StackOverflow where a workaround was provided. IMHO this should simply be fixed instead.

To Reproduce Steps to reproduce the behavior:

1. Start the example app
2. Edit a book, make the field “author_email” empty.
3. Export all books as .xlsx
4. Import this .xlsx
5. sqlite3.IntegrityError: NOT NULL constraint failed: core_book.author_email

Versions (please complete the following information):

• Django Import Export: [2.8.0]

Expected behavior I expect no error. In particular: I expect that the output of an export is a valid input for an import.

[pokken-magic]

Yeah, I think we essentially need to inspect both Null and Blank setting on the django field and then set those values appropriately. Then when saving, we need to interpret that correctly. And we can allow users to explicitly override that at the field level (The way they can say saves_null_values, they could also have saves_blank_values)

My thinking is:

-if blank and null are true, empty cells are “” -if blank is true and null is false, empty cells are “” -if blank is false and null is false, empty cells are None, and will fail -if blank is false and null is true, empty cells are None

I think the things we need to do are:

1. update field constructor to take saves_blank_values with a default of False
2. update field.save to know what to do based on saves null / saves blank
3. update field detection to correctly inspect the django field for saves_blank and saves_null

The problem here is that this reduces a little bit of functionality, which is that dialects that have a clear distinction between “” and null (json, yaml) won’t be able to have both blanks and nulls.

I can’t think of any way to get around that except to move things even further up the chain to the importer, but the importer is not aware of the django field. so problematic there.

It might make some sense long term to think about the widget getting information about the blank/null status tbh. I’ll have to think about this some more.

[matthewhegarty]

Thanks for working through this issue.

I think I remember that the issue is that Excel saver interprets “” as an empty cell, which the importer interprets as None?

I tested and in this case the author_email is exported as empty string “”. In fact, ‘None’ values are always forced to strings on export - this has been in the library from the start.

So in the case of Excel, we are exporting as empty string. In Excel, I couldn’t find a way to explicitly set a cell as being an empty string. I tried setting the cell as type ‘Text’ and importing, but it is always interpreted as a ‘None’ (null) value by tablib when importing. So we have the situation where we always export as empty string, but always get back a None value.

To fix this (rather than having to rely on the users using the workaround) I think we would have to include the logic you describe here. Namely that the import process has to be more explicit when dealing with non-nullable fields.