Persisting Nested Objects

Note: Cross-posting from http://discuss.elastic.co. Not sure where is better for elasticsearch-dsl questions.

I’m attempting to persist an object to elasticsearch with a nested object. However instead of using syntax like:

child = field.Object(properties={'name': field.String()})

I’d like to have child be it’s own object.

from elasticsearch_dsl import field, document
from elasticsearch_dsl import connections
connections.connections.create_connection(hosts=['localhost'], timeout=20)


class Child(field.Object):
    name = field.String()


class Parent(document.DocType):
    name = field.String()
    child = Child()

    class Meta:
        index = 'myindex'

parent = Parent(name='test parent')
parent.child = Child(name='test other child')
parent.save()

However, doing it this way results in:

  File "/home/abarilla/PycharmProjects/iocdb/iocdb/model/test.py", line 25, in <module>
    parent.child = Child(name='test other child')
  File "/home/abarilla/.virtualenvs/iocdb/local/lib/python2.7/site-packages/elasticsearch_dsl/document.py", line 115, in __setattr__
    return super(DocType, self).__setattr__(name, value)
  File "/home/abarilla/.virtualenvs/iocdb/local/lib/python2.7/site-packages/elasticsearch_dsl/utils.py", line 418, in __setattr__
    value = self._doc_type.mapping[name].to_python(value)
  File "/home/abarilla/.virtualenvs/iocdb/local/lib/python2.7/site-packages/elasticsearch_dsl/field.py", line 63, in to_python
    return self._to_python(data)
  File "/home/abarilla/.virtualenvs/iocdb/local/lib/python2.7/site-packages/elasticsearch_dsl/field.py", line 133, in _to_python
    return self._doc_class(self.properties, **data)
TypeError: type object argument after ** must be a mapping, not Child

Is there a way I can accomplish this?

[JoshCoady]

I wasn’t sure if this warranted a new issue, but along these same lines, it seems like currently you need to define your classes like this:

class OrderItem(InnerObjectWrapper):
    pass

class Order(DocType):
    items = Object(
        doc_class=OrderItem,
        multi=True,
        properties={
            'description': Text(),
            'quantity': Integer(),
            'price': Float()
        }
    )

You don’t actually need the OrderItem class if you aren’t going to customize it. You could just set doc_type=InnerObjectWrapper, but I included it this way so that the examples are consistent.

Where I’d prefer to be able to define them more like this:

class OrderItem(InnerObjectWrapper):
    description = Text()
    quantity = Integer()
    price = Float()

class Order(DocType):
    items = Object(
        doc_class=OrderItem,
        multi=True
    )

In this way the properties definitions for inner objects is consistent with that of the top level. It also allows you to keep your field definitions with the object it belongs to, reducing coupling and allowing for inner objects that could be reusable without having to duplicate the properties definition in every using class.

For example, say you have a CMS with an index for each type of content like articles, slideshows, and quizzes, etc. But each of those documents may want to have an author sub object. If the properties are set where they naturally belong (on the author class) then you don’t need do duplicate that code to all of the different content type doc classes.

class Author(InnerObjectWrapper):
    name = Text()
    image_url = Text()
    bio_url = Text()

Is there maybe a way to do this that I’m missing or is it on the roadmap or do I just need to use properties for inner objects?

[gilbsgilbs]

I agree with @JoshCoady . The current way to define Nested objects is quite inconsistent and counter-intuitive. It also prevents reusability of nested objects. I think there should be virtually no difference between nested objects and inner objects from the ORM perspective, apart from the way you declare them:

class OrderItem(InnerObject):
    description = Text()
    quantity = Integer()
    price = Float()

class Order(DocType):
    items = Nested(doc_class=OrderItem)
    # or
    items = OrderItem(multi=True)

Let’s say I want to denormalize the last item, I’d like to be able to do something like this:

class Order(DocType):
    items = Nested(doc_class=OrderItem)
    last_item = OrderItem()

Currently, I have to duplicate the OrderItem object to be able to do this, and this is very ugly. Or I missed something?