Should EmbeddedDocument child of abstract EmbeddedDocument have a cls field?

EmbeddedDocument and Document behaves differently with regards to the cls field:

• If ChildDocument inherits from AbstractDocument, it has no cls field.
• If ChildEmbeddedDocument inherits from AbstractEmbeddedDocument, it gets a cls field.

This results from a difference in the code:

def _is_child(bases):
    """Find if the given inheritance leeds to a child document (i.e.
    a document that shares the same collection with a parent)
    """
    return any(b for b in bases if issubclass(b, DocumentImplementation) and not b.opts.abstract)

def _is_child_embedded_document(bases):
    """Same thing than _is_child, but for EmbeddedDocument...
    """
    return any(b for b in bases
               if issubclass(b, EmbeddedDocumentImplementation) and
               b is not EmbeddedDocumentImplementation)

I’m not sure I understand the rationale behind this.

We want to create an abstract parent EmbeddedDocument in our app with shared fields and methods for our EmbeddedDocuments to inherit, and now we get cls fields in every EmbeddedDocument, which pollute the API output. It would be easier to deal with this if it only happened when a concrete EmbeddedDocument is subclassed (which is much less frequent).

It’s easy not to add cls from a concrete EmbeddedDocuments when its parent is abstract. Just copy Document:

def _is_child_embedded_document(bases):
    """Same thing than _is_child, but for EmbeddedDocument...
    """
    return any(b for b in bases
               if issubclass(b, EmbeddedDocumentImplementation) and not b.opts.abstract)

but if we do that, we should also prevent EmbeddedField from accepting an abstract EmbeddedDocument, otherwise, it won’t be able to deserialize that concrete child.

Is this the reason you did things this way?

It boils down to what we expect from an abstract EmbeddedDocument.

• For real polymorphism, where the abstract EmbeddedDocument is Vehicle and its children are Car and Truck, it makes sense to keep things the way they are. We may want to create an EmbeddedField that expects a concrete Vehicle subclass but not a Vehicle.

• If we create a SuperEmbeddedDocument for our EmbeddedDocuments to inherit, it does not make much sense to accept it in EmbeddedField and I don’t see the use for the cls field.

It looks like we’re dealing with two different concepts. Are we abusing inheritance by doing this? Should we do it differently? Should there be two different levels of abstraction?

Notes:

• If we keep things this way, we may need to add a protection in EmbeddedField._deserialize because IIUC, if value has no cls key, it will try to instantiate the parent abstract EmbeddedDocument, which will trigger umongo.exceptions.AbstractDocumentError: Cannot instantiate an abstract EmbeddedDocument rather than a ValidationError (I could be wrong, I didn’t test that).

• Such a restriction applies to Document: you can’t reference an abstract VehicleDocument since it has no collection.

• I don’t see any simple way to prevent the API from spitting those cls fields out. Any nested structures with a little bit of complexity would require awful params trickery in as_marshmallow_schema.

[touilleMan]

Don’t worry for the mess 😉

I think the difference between embedded document and document is due to the fact a document is linked with a collection:

• Abstract document ==> no collection, cannot be instantiated
• Regular document ==> collection
• Regular Child document ==> same collection than it parent, get a cls field as discriminent

On the other hand, and embedded document has nothing to do with collections. However given it implementation comes from the document’s one (doctstring of EmbeddedDocumentOpts.abstract shows this for example ^^), it get the abstract system.

I tried to fix what you described (see f834177514e7a56772c443c042e2e07139ea8ebb):

• Abstract embedded document and it direct children don’t provide cls field
• Abstract embedded document cannot be used in EmbeddedField

However I’m not sure this is the right way to do: is abstract really meaningful for embedded document ? Without abstract, you would just define regular class (not EmbeddedDocument) to make my abstract inheritances, then use multiple inheritance to create concrete class.

This is typically what I would do to solve your current usecase btw:

 class TimedEmbeddedDocumentMixin:
    created_at = DateTimeField()
    updated_at = DateTimeField()

class MyEmbeddedDoc(TimedEmbeddedDocumentMixin, EmbeddedDocument):
    foo = IntField()

So maybe we could just put a word about this pattern in the EmbeddedDocument documentation and drop the abstract stuff from it.

What do you think ? Do you see a special usecase that would require abstract for the embedded document ?

[lafrech]

Regarding the removal of cls field when dumping mentioned in OP, it is more or less unrelated.

I use a custom schema for this right now.

class ExtraSchema(SchemaFromUmongo):
    """Custom Marshmallow Schema for our Schemas"""

    # XXX: This kinda sucks. Deal with it in umongo?
    def _init_fields(self):
        super()._init_fields()
        self.fields.pop('cls', None)
        self.load_fields.pop('cls', None)
        self.dump_fields.pop('cls', None)

This might be worth a umongo feature…