Some subspaces missing from the directory of very large Spaces over federation

Description

When opening up the room directory for a very large space such as #community:matrix.org from a federating homeserver (matrix.apetre.sc in my case, but I’ve reproduced it from a few others as well), only a subset of the subspaces are visible.

Steps to reproduce

• Open the room directory of #community:matrix.org from a homserver other than matrix.org
• I’d expect to see 38 rooms and 11 spaces, but instead I see 54 rooms and 3 spaces:

Other non-matrix.org users on the Spaces feedback room have experienced the same issue with the exact same room/subspace counts that I was seeing, so it’s unlikely to be just a cached bad value.

Clearing cache and reloading does not help. Trying with different instances of Element does not help either.

Version information

• Homeserver: matrix.apetre.sc

If not matrix.org:

• Version: v1.36.0

• Install method: Docker

• Platform: Linux

[clokep]

MSC2946 is the canonical place to track this work btw. We first need to finish the design, then implement it.

[clokep]

I’m going to close this in favor of #10495, which includes pagination.