Defining fields programmatically

I have a list of field names as strings

FIELDS = ('field_1', 'field_2',..., 'field_42')

and I would like to create this schema programmatically:

class MySchema(Schema):

    field_1 = Float()
    field_2 = Float()
    ...
    field_42 = Float()

I managed to do that with a custom metaclass in which one can specify lists of dynamic fields. Each element of the list is a tuple containing both a list of field names and a Field specification (a dict containing field class + optional args and kwargs).

    class DynamicSchemaOpts(SchemaOpts):
    
        def __init__(self, meta):
            super().__init__(meta)
            self.auto_fields = getattr(meta, 'auto_fields', [])
    
    
    class DynamicSchemaMeta(SchemaMeta):
    
        @classmethod
        def get_declared_fields(mcs, klass, cls_fields, inherited_fields, dict_cls):
    
            fields = super().get_declared_fields(klass, cls_fields, inherited_fields, dict_cls)
    
            for auto_field_list in klass.opts.auto_fields:
                field_names, field = auto_field_list
                field_cls = field['cls']
                field_args = field.get('args', [])
                field_kwargs = field.get('kwargs', {})
                for field_name in field_names:
                    fields.update({field_name: field_cls(*field_args, **field_kwargs)})
    
            return fields
    
    
    class MySchema(Schema, metaclass=DynamicSchemaMeta):
    
        OPTIONS_CLASS = DynamicSchemaOpts
    
        class Meta:
            strict = True
            auto_fields = [
                (FIELDS,
                 {'cls': Float}),
            ]

See this SO question for the whole story and an example using a field with args and kwargs.

It seems to work.

I’d appreciate some feedback. Is this the right way to address this issue?

The code would be simpler if I could pass a field instance (especially for fields with args and kwargs)

class Meta:
    strict = True
    auto_fields = [
        (FIELDS, Float()),
    ]

but then all the fields in FIELDS would share the same Nested field instance. Could this be an issue? I peaked into the fields in Marshmallow and didn’t identify any use case that would break, but it doesn’t look like a good idea. A custom field could store some information in its instance that is not meant to be shared. I’d appreciate feedback about this as well.

I thought that would be a Python issue with a generic Python solution, but what I did is specific to Marshmallow. Would it make sense to add this feature to the library?

[maximkulkin]

What happened to plain old metaprogramming?!

MySchema = type('MySchema', (marshmallow.Schema,), {
    attr: marshmallow.fields.Float()
    for attr in FIELDS
})

You can even have different types of fields there:

fields = {}
fields['foo'] = marshmallow.fields.Float()
fields['bar'] = marshmallow.fields.String()
MySchema = type('MySchema', (marshmallow.Schema,), fields)

or as a base for your customizations:

class MySchema(type('_MySchema', (marshmallow.Schema,), fields)):
    @marshmallow.post_dump
    def update_something(self, data):
        pass

[maximkulkin]

You could also do

MySchema = type(
    'MySchema',
    (marshmallow.Schema,),
    dict({
        field: ma.fields.Nested(MyEmbeddedSchema)
        for field in FIELDS
    }, Meta=type('Meta', tuple(), {'strict': True})
)