Creating additional fields on the fly

I’m trying to create a very flexible serializer, such that users can generate additional fields in the future. Let’s say that today they only need the defaults I’ve provided

class PostSerializer(Serializer):
    id = fields.String()
    title = fields.String(default="Untitled")
    body = fields.String(default=None)
    author = fields.List(fields.String) 

The user creates several posts, and they decide they want a field for “category.” I provide an interface where they set a new category field. Now perhaps I store this field in a dictionary.

additional_fields = {
    "category" : "list"
}

When I modify the serializer on the fly (the only way that seems to work is via Meta.additional, setattr never seems to work)

s = PostSerializer
PostSerializer.Meta.additional = additional_fields.keys()

Posts which were created without the ‘category’ field will cause the following AttributeError:

AttributeError: "category" is not a valid field for {'id': '123456', 'title': 'Cool Post', 'body': 'Lorem Ipsum...', 'author': ['John', 'Steve']}

How can I maintain flexibility to add user generated fields, but also protect myself in the future? Is there a way to set a global default for additional fields?

[sloria]

@skqr You can use a @post_load method to do the same thing as data_handler.

Yet another alternative is to update a schema’s fields on __init__.

from marshmallow import Schema, fields

class MySchema(Schema):

    def __init__(self, additional_fields=None, **kwargs):
        super().__init__(**kwargs)
        self.declared_fields.update(additional_fields)

additional_fields = {
    'foo': fields.Int()
}

sch = MySchema(additional_fields=additional_fields)

print(sch.dump({'foo': '123'}).data)  # {'foo': 123}

[maximkulkin]

@MrYoda Here is an example of how to do that (if I understand problem correctly):

import marshmallow as m
import marshmallow.fields as mf
from marshmallow.compat import with_metaclass
from collections import namedtuple, OrderedDict

LANGUAGES = ['en', 'de', 'fr']

class I18NMeta(type):
    def __new__(cls, name, bases, attrs):
        new_attrs = OrderedDict(attrs)
        if 'Meta' in attrs:
            for name in getattr(attrs['Meta'], 'i18n_fields', []):
                if name not in attrs:
                    continue

                field = attrs[name]
                for lang in LANGUAGES:
                    new_attrs['%s_%s' % (name, lang)] = field

        return type(name, bases, new_attrs)

class ModelSchema(with_metaclass(I18NMeta, m.Schema)):
    class Meta:
        ordered = True
        i18n_fields = ['value']

    value = mf.String()

Model = namedtuple('Model', ['value', 'value_en', 'value_de', 'value_fr'])

print ModelSchema().dump(Model('Hello', 'Hello', 'Hallo', 'Bonjour'))
# => MarshalResult(data=OrderedDict([(u'value', u'Hello'), (u'value_en', u'Hello'), (u'value_de', u'Hallo'), (u'value_fr', u'Bonjour')]), errors={})

Alternatively, you can use special type to mark localized strings:

import marshmallow as m
import marshmallow.fields as mf
from marshmallow.compat import with_metaclass, iteritems
from collections import namedtuple, OrderedDict

LANGUAGES = ['en', 'de', 'fr']

class LocalizedString(mf.String):
    pass

class I18NMeta(type):
    def __new__(cls, name, bases, attrs):
        new_attrs = OrderedDict(attrs)
        for field_name, field in iteritems(attrs):
            if not isinstance(field, LocalizedString):
                continue

            for lang in LANGUAGES:
                new_attrs['%s_%s' % (field_name, lang)] = field

        return type(name, bases, new_attrs)

class ModelSchema(with_metaclass(I18NMeta, m.Schema)):
    value = LocalizedString()

Model = namedtuple('Model', ['value', 'value_en', 'value_de', 'value_fr'])

print ModelSchema().dump(Model('Hello', 'Hello', 'Hallo', 'Bonjour'))