lost data when use to_dict() to result

{
  "_source": {
    "base_info": {
      "profile": null,
      "name": null,
      "headline": null,
      "industry": null,
      "avatar": null,
      "email": "******@gmail.com",
      "location": null
    }
}

There is the data I have stored in ES and I can get them with RESTful API . The API returns me the full text .
When I get the doc use elasticsearch-dsl-py I also can get the data in search_result<response> like this: {'location': None, 'name': None, 'avatar': None, 'industry': None, 'profile': None, 'headline': None, 'email': '*****@gmail.com'}. But when I try to use result_list = [item.to_dict() for item in search_result] I find the dict it returned is {'email': '******@gmail.com' } and the items which values = None has gone away!

[honzakral]

implemented in 0ee7feeadcc1fdcdb0817e5e0c6816c5f9cff728

[honzakral]

I understand your issue. The reason why we decided to ignore those empty fields is:

• from the dsl point of view there is no difference - accessing tags will return an empty list if not present on the document, accessing body would return None.
• from the point of view of elasticsearch, there is again no difference
• finally, sometimes we create those values as a side effect (like accessing tags on an empty document) and we don’t want those to then bloat the document and the index

that said we could totally include a flag to control that behavior for when the data is shared with other systems that don’t have similar properties.