How might one go about defining a StreamField with a dynamic set of blocks?

My wagtail server needs to to be able to restrict specific StreamField blocks to specific Sites. My initial idea for implementing this was to do use wagtail-flags to prevent specific blocks from being rendered in the StreamField form unless the current Site has the right flag set.

However, our main FlexPage StreamField is monumentally complicated. You can put instances of Blocks B, C, D, and Q inside instances of Blocks A, R, and/or Z, which can be inside instances of Block D and/or E (only it’s more complicated than that). The massive combinatorial nature of this setup causes our StreamField’s definition to be supremely unwieldy, such that the AlterField migrations django creates for this field are upwards of 400,000 characters on a single line, even when we change just a single setting on a single sub-Block.

Attempting the wagtail-flags approach I mentioned above caused nearly 5000 SQL queries to be issued every time the StreamField’s form was rendered, because the stream_menu.html template is rendered several hundred times for our form. This is obviously untenable, so I’m currently brainstorming an alternative.

The best idea I’ve come up with is to re-imagine the way that a StreamField defines its child block set. Instead of defining it at import, I’d like for the StreamField to define it’s available child blocks at runtime. That will let me check the current Site during the request, adding blocks to the list only if they have an appropriate flag. If this is possible, it should actually kill two birds with one stone: django will no longer need to generate massive, pointless migrations (from the DB’s point of view, StreamField is really just a TextField), and we can restrict the available blocks at a very low level, requiring much fewer SQL queries.

Do you guys think this would be possible? I’m currently researching this idea, delving deep into StreamField and StreamBlock, in the hopes of finding a way to do this. Any insight from anyone who actually knows this code would be greatly appreciated.

[coredumperror]

What we ended up doing was to subclass StreamBlock and StreamField like so:

class CustomStreamBlock(blocks.StreamBlock):
    """
    Identical to StreamBlock, except that we override the constructor to make it save self._base_blocks and
    self._dependencies, instead of self.base_blocks and self.dependencies. This lets us replace them with @properties.
    """

    def __init__(self, local_blocks=None, **kwargs):
        self._constructor_kwargs = kwargs

        # Note, this is calling BaseStreamBlock's super __init__, not FeatureCustomizedStreamBlock's. We don't want
        # BaseStreamBlock.__init__() to run, because it tries to assign to self.child_blocks, which it can't do because
        # we've overriden it with an @property. But we DO want Block.__init__() to run.
        super(BaseStreamBlock, self).__init__(**kwargs)

        # create a local (shallow) copy of base_blocks so that it can be supplemented by local_blocks
        self._child_blocks = self.base_blocks.copy()
        if local_blocks:
            for name, block in local_blocks:
                block.set_name(name)
                self._child_blocks[name] = block

        self._dependencies = self._child_blocks.values()

    @property
    def child_blocks(self):
        request = get_current_request()
        # Protect against crashing in case this ever runs outside of a request cycle.
        if request is None:
            return self._child_blocks
        return OrderedDict([
            item for item in self._child_blocks.items() if item[0] not in request.site.features.disabled_defaults
        ])

    @property
    def dependencies(self):
        request = get_current_request()
        # Protect against crashing in case this ever runs outside of a request cycle.
        if request is None:
            return self._child_blocks
        return [block for block in self._dependencies if block.name not in request.site.features.disabled_defaults]


class CustomStreamField(StreamField):

    def __init__(self, block_types, **kwargs):
        super(StreamField, self).__init__(**kwargs)
        if isinstance(block_types, Block):
            self.stream_block = block_types
        elif isinstance(block_types, type):
            self.stream_block = block_types(required=not self.blank)
        else:
            self.stream_block = CustomStreamBlock(block_types, required=not self.blank)

This makes it so the attributes which the rest of the StreamField/StreamBlock code read from are alterable at runtime based on outside configuration. You can implement the dependencies and child_blocks properties to work with whatever outside configuration you decide to use. I implemented a mechanism that stores lists of custom Block class names on a Site-by-Site basis, and excludes from the @propertys any classes with matching names. This lets superusers “disable” certain blocks for certain Sites.

With this technique, just make sure that all your code uses your child classes wherever it had previously been using StreamBlock and StreamField, and define your CustomStreamField instances using the “list of (block_name, Block instance) tuples”, so that the else block in your CustomStreamField subclass triggers.

This doesn’t solve the problem of ultra-bloated migrations, but it works well enough for our purposes.

[coredumperror]

more like reinventing it as a totally different kind of field

Not a bad idea, actually. Having an entirely separate version of StreamField that’s designed from the ground up with dynamism in mind would actually solve both of the problems that my team (and several others who’ve piped up in various issues around here) have had with the current implementation of StreamField. Namely: dynamic block choices and better handling of complex block combinatorics.

I’d love to delve into this myself, but my team is severely overworked, and I simply don’t have time to dedicate a week or so to an experiment of this scale. Maybe some time next year I’ll have enough free cycles to look into this further.