Bug in handling of z_index

Here’s a Scene

from manim import *

class SquareToCircle(Scene):
    def construct(self):
        circle = Circle().set_fill(RED, opacity=1)
        square = Square(side_length=1.7).set_fill(BLUE, opacity=1)
        square.z_index = circle.z_index - 1

        self.play(FadeIn(VGroup(circle, square)))  # all good
        self.play(ApplyMethod(circle.shift, UP))   # woops!
        self.wait(1)

And here’s the output:

As you can see, the square comes on top of the circle at the end of the second animation. I was expecting the square to remain behind the circle throughout the video because of the line square.z_index = circle.z_index - 1. Is this intended behavior? Is this a known issue?

FWIW, I think currently there are two different systems living in manim. First, the Scene and Camera classes try to keep the mobjects in order, and then the order of rendering on screen is (or, used to be) taken from that order. But after the introduction of z_index in #122, all of that bookkeeping to try to maintain the order is unnecessary since at the last minute the mobjects are being sorted by their z_index. I believe it is the interaction of these two things that is causing the bug. We can perhaps take this opportunity to deprecate all of those bookkeeping operations and stick to the z_index.

cc @Tony031218 as author of #122

[leotrs]

@Tony031218 , thanks for looking into this and for providing alternatives. Something tells me that we’ll have to think long and hard about this one!

[behackl]

By changing extract_mobject_family_members to sort all extracted mobjects, rather than just the passed ones (which doesn’t affect mobjects which are contained within groups), i.e.

diff --git a/manim/utils/family.py b/manim/utils/family.py
--- a/manim/utils/family.py
+++ b/manim/utils/family.py
@@ -28,6 +28,7 @@ def extract_mobject_family_members(
         method = Mobject.family_members_with_points
     else:
         method = Mobject.get_family
+    extracted_mobjects = remove_list_redundancies(list(it.chain(*[method(m) for m in mobjects])))
     if use_z_index:
-        mobjects = sorted(mobjects, key=lambda m: m.z_index)
-    return remove_list_redundancies(list(it.chain(*[method(m) for m in mobjects])))
+        return sorted(extracted_mobjects, key=lambda m: m.z_index)
+    return extracted_mobjects

I get for @leotrs’s example from the original post. Which looks fine to me (?)

What is the drawback of reordering the list of extracted mobjects? The sorting is stable, i.e., doesn’t change the order of mobjects with the same z_index, and it is not like the list returned by extract_mobject_family_members is really used by anything non-rendering related (at least it didn’t seem otherwise after a quick search for extract_mobject_family_members – I could be wrong though). Anyways: locally, our current test suite also passes with this change.