0.5.1 Breaks tests

After upgrading from 0.5.0 -> 0.5.1, it looks like the value in the snapshot is being returning as non-serializable generator?

    def assert_equals(self, value, snapshot):
>       assert value == snapshot
E       assert <[TypeError("...x7f40bc5cd3f8> == [{'geometry': ...': 'Feature'}]
E         (pytest_assertion plugin: representation of details failed.  Probably an object has a faulty __repr__.)
E         /home/app/.local/share/virtualenvs/app-4PlAip0Q/lib/python2.7/site-packages/simplejson/encoder.py:273: TypeError: Object of type generator is not JSON serializable

E       assert <[TypeError("...x7f40c119aab8> == {'features': [...reCollection'}
E         Omitting 1 identical items, use -vv to show
E         Differing items:
E         {'features': <generator object <genexpr> at 0x7f40bc63fb90>} != {'features': [{'geometry': {'coordinates': [-0.077092051506, 51.5275860195], 'type': 'Point'}, 'properties': {'name': ..., ...], 'type': 'LineString'}, 'properties': {'distance': 608.4, 'duration': 60.0, 'name': 'Raw'}, 'type': 'Feature'}]}
E         Use -v to get the full diff

Anyone found a fix?

[medmunds]

@StoicLoofah thanks for tracking this down.

I think there is a snapshottest bug or mis-feature in CollectionFormatter.normalize:

class CollectionFormatter(TypeFormatter):
    def normalize(self, value, formatter):
        iterator = iter(value.items()) if isinstance(value, dict) else iter(value)
        # ... This line is the problem:
        return value.__class__([formatter.normalize(item) for item in iterator])

This is going to have problems with all collection subclasses that require additional parameters in their constructors. (And indeed, there’s already an override for defaultdict.)

At a minimum, CollectionFormatter.normalize should probably catch exceptions trying to re-construct a normalized object, and issue a more helpful error message (suggesting the user either convert the snapshotted object to a simpler type, or that they add a custom formatter). Ideally, snapshottest would also preflight this while writing snapshot files, so that the error is reported then, not only when comparing snapshots later.

But I’m wondering if a better fix might be to change the value.__class__(...) line to instead construct the base type the formatter writes into snapshots. E.g., CollectionFormatter(dict, format_dict).normalize(value) should just try to construct a plain dict—even if value is a DRF ReturnDict—because format_dict always just writes snapshots as plain dicts. Then we wouldn’t need all these extra custom formatters: if a formatter is willing to write a snapshot, then it really should be able to compare the same type of object to that snapshot later.

(I confess I’m getting a little lost in understanding the need for this normalize logic in the first place. It transforms the actual value being compared (not the expected snapshot value) prior to comparison. I think this was added in #82 to fix problems introduced in a bunch of other major changes between v0.5.0 and v0.5.1.)

Also, it seems like the original report from @hobochild is probably related, but I think that specific case might be covered by the fix in #116 that went into v0.6.0.

[StoicLoofah]

I was able to fix it by adding the following into my conftest.py

from rest_framework.utils.serializer_helpers import ReturnDict, ReturnList
from snapshottest import formatter, formatters

class DrfReturnCollectionFormatter(formatters.TypeFormatter):
    def normalize(self, value, formatter):
        iterator = iter(value.items()) if isinstance(value, dict) else iter(value)
        # Added the serializer kwarg as required by ReturnDict or ReturnList
        return value.__class__([formatter.normalize(item) for item in iterator],
                               serializer=None)

formatter.Formatter.register_formatter(DrfReturnCollectionFormatter(ReturnDict,
                                                                    formatters.format_dict))
formatter.Formatter.register_formatter(DrfReturnCollectionFormatter(ReturnList,
                                                                    formatters.format_list))

Not sure how you guys want to handle this since it’s an upstream issue but figured I would let you know how I addressed it!