Empty strings as None (revisited)

I’d like to revisit this suggestion - allowing empty strings to be interpreted as None:

https://github.com/marshmallow-code/marshmallow/issues/543

I understand the sentiment of the response: “it’s a bad idea to support that”. However, that’s not sufficient for some use cases.

For example, right now I’m trying to use marshmallow to convert an XML format where null values are sent as empty strings. I don’t get to control the XML format.

I think it’s entirely reasonable for a Field to interpret a null datestring (or a null decimal string) as None, and control via allow_none whether to pass that. I don’t think it’s reasonable to fall down and say “the NULL value defined by your format is invalid, and we’re not going to allow you to express that”. This just forces a lot of ugly preprocessing workarounds onto library users, who need to clutter their code with superfluous, bug-prone value = value or None type declarations… thereby mixing format definition code in the middle of application code, which makes it hard to use marshmallow to write good code for these use cases.

I feel that the right place to define a format is, well, in the Schema class definition. A patch to accomplish this is simple:

• add an attribute to fields.Field, e.g. null_values = {None}
• change fields.Field.deserialize() line 263 as follows: if getattr(self, 'allow_none', False) is True and value in self.null_values: return None

We can stop there, and allow users to support their nasty unhygienic formats by overriding null_values in a custom subclass… or even take it a step further and allow first-class support for text-only formats by directly setting null_values = {None, ''} for fields.Decimal, fields.DateTime, fields.Date, fields.Time, etc. By Zeus, maybe even fields.Integer could treat empty strings as None!!

I’m happy to submit a PR to that effect, but I’d like to see where the developers’ heads are at vis-a-vis using marshmallow for non-JSON formats before wasting anybody’s time.

Thanks, and happy new year!

[sloria]

Actually, I think I was misunderstanding the OP. IIUC, the goal is to interpret "" as missing and deserialize to None.

I think a more direct and flexible way to achieve this would be to allow specifying the values that should be interpreted as missing (as suggested by @lafrech in a previous comment).

# proposed API

class ArtistSchema(Schema):
    # if input is None, "", or is missing from the input, deserialize to None
    name = fields.Str(missing=None, missing_values={None, ""})

print(ArtistSchema().load({"name": ""}))  # {'name': None}

If instead your backend models "" as missing, you just change the value of missing.

class ArtistSchema(Schema):
    name = fields.Str(missing="", missing_values={None, ""})

This would allow any field to define what it means to be required. For example:

class ArtistSchema(Schema):
    album_names = fields.List(fields.Str(), required=True, missing_values=([], ()))


ArtistSchema().load({"album_names": []})
#  ValidationError: {'album_names': ['Missing data for required field.']}

What do you think?

[sersorrel]

This would be really nice for parsing e.g. dates from HTML forms, where an empty <input type=date> (or any empty <input>) results in the empty string.