@post_load function not invoked if data dict is updated from @pre_load?

Hi guys,

I’m new to marshmallow and I’m trying to de-serialize JSON data where any key can potentially have a ‘None’ string value (in place of the none json type)

I’d like to set these to Python’s None type and after carefully reading the docs it seems like @pre_load would be the way to do this. I’d like to return a POPO as a result, so I also have a @post_load for that.

The problem I’m experiencing is that, once I make an update to the data dict in @pre_load, I get a <type ‘dict’> back in result instead of an instance of class User.

If @pre_load doesn’t touch the data (for example, if we pass in user_json2 which has no ‘None’ strings) then the result is an instance of class User. As soon as we update a key, we’re getting back a dictionary. It would appear that post_load is not being invoked.

Here’s the example code-

#Our Plain Old Python Object (POPO)
 class User(object):
     def __init__(self, **kwargs):
         for key, value in kwargs.iteritems():
             setattr(self, key, value)

 #Marshmallow Schema
 from marshmallow import Schema, fields, pre_load, post_load
 class UserSchema(Schema):
     firstName = fields.Str(load_from='FIRST_NAME')
     middleNames = fields.Str(load_from='MIDDLE_NAMES')
     lastName = fields.Str(load_from='LAST_NAME')
     fullLegalName = fields.Str(load_from='FULL_LEGAL_NAME')
     knownAs = fields.Str(load_from='KNOWN_AS')
     emailAddress = fields.Str(load_from='EMAIL_ADDRESS')


     @pre_load
     def NoneToNoneType(self, data):
         for key, value in data.iteritems():
             if value == 'None':
                 data[key] = None
         return data

     @post_load
     def OutputDomainObject(self, data):
        #This print statement is never shown if user_json is passed in...
         print('post_load invoked!')
         return User(**data)

 user_json = '{"FIRST_NAME": "Kali","LAST_NAME": "Subra","KNOWN_AS": "None","FULL_LEGAL_NAME": "Kali Subra","EMAIL_ADDRESS": "None","MIDDLE_NAMES": "None"}'
 user_json2 = '{"FIRST_NAME": "Kali","LAST_NAME": "Subra","KNOWN_AS": "Z","FULL_LEGAL_NAME": "Kali Subra","EMAIL_ADDRESS": "Z","MIDDLE_NAMES": "Z"}'    

 user_schema = UserSchema()
 result = user_schema.loads(user_json)

[ptdel]

@deckar01 I was about to say “prepare to fight” because I just ran yesterday, but as I run it again from the REPL:

>>> row = {
...   'Id': 1, 'Company': 1, 'LastModified': '2018-04-09T18:14:32',
...   'Service': 'EC2', 'Name': 'Running Instances', 'Value': '4',
...   'Description': 'Across 8 Regions',
...   'DateOfResults': '2014-08-18T20:31:21'
... }
>>> item = InventorySchema().load(row)
>>> item
<__console__.Inventory object at 0x7f551f3f4da0>
>>> try:
...     item2 = InventorySchema().load(row)
... except ValidationError as e:
...     print(e)
...
...
>>> item2
<__console__.Inventory object at 0x7f551f7c85c0>
>>>

I have no idea what happened here but it must be something transient that I managed to bork. Disregard.

[deckar01]

I ran the example with 3.0.0b8 and did not get the reported issue.

from marshmallow import Schema, fields, post_load, post_dump

class Inventory:
    def __init__(self, **data):
        pass

class InventorySchema(Schema):
    Id = fields.Int()
    Company = fields.Int()
    LastModified = fields.DateTime()
    Name = fields.Str()
    Value = fields.Str()
    Description = fields.Str()
    Service = fields.Str()
    DateOfResults = fields.DateTime()
    
    @post_load
    def toModel(self, data):
        return Inventory(**data)
    
    @post_dump
    def toDict(self, data):
        return dict(data)

row = {
  'Id': 1, 'Company': 1, 'LastModified': '2018-04-09T18:14:32',
  'Service': 'EC2', 'Name': 'Running Instances', 'Value': '4',
  'Description': 'Across 8 Regions',
  'DateOfResults': '2014-08-18T20:31:21'
}

item = InventorySchema().load(row)

item
# <__main__.Inventory object at 0x1045a7400>