[BUG] spawn/list with many prototypes blocks or hangs server until memory exhausted

Describe the bug

Where many prototypes exist (~1000), when the spawn/list command is issued, the server pauses for a considerable amount of time.

At 5000 or more, the server takes longer and longer to complete the command to the point where memory is more likely to be exhausted than for the result to be returned. (Unlike #1994 the usage does not just skyrocket-- it builds gradually).

To Reproduce

from evennia.utils.test_resources import EvenniaTest
from random import sample
from evennia.prototypes import prototypes as protlib
import uuid
from time import time


class PrototypeCrashTest(EvenniaTest):
    
    num_prototypes = 1000
    
    def create(self, num=None):
        if not num: num = self.num_prototypes
        print(f"Creating {num} prototypes...")
        for x in range(num):
            prot = {
                'prototype_key': str(uuid.uuid4()),
                'some_attributes': [str(uuid.uuid4()) for x in range(10)],
                'prototype_tags': list(sample(['demo', 'test', 'stuff'], 2)),
            }
            protlib.save_prototype(prot)
    
    def test_prototype_dos(self, *args, **kwargs):
        num_prototypes = self.num_prototypes
        for x in range(10):
            self.create(num_prototypes)
            print(f"Attempting to list prototypes...")
            start_time = time()
            self.char1.execute_cmd('spawn/list')
            print(f"Prototypes listed in {time()-start_time} seconds.")
            num_prototypes = num_prototypes * x

Expected behavior

A quicker result would be more favorable.

Environment, Evennia version, OS etc

    Evennia 0.9.0 (rev a6ed068) (rev a6ed068)
    OS: posix
    Python: 3.7.7
    Twisted: 19.10.0
    Django: 2.2.12

Additional context

As with #1994 I think the problem(s) in this case are again the entire queryset being loaded into memory instead of using .iterator():

https://github.com/evennia/evennia/blob/6c968af31326bb034e7a5d7ae6e9fa314102562d/evennia/prototypes/prototypes.py#L383-L393

But that may not even be necessary either; for L393 in particular-- surely grabbing the value of a single attribute could be better performed by something like the Django values_list or only filters to do the work in SQL instead of manually iterating the queryset in Python? I think it is this line that forces evaluation of the entire queryset anyway.

This much data being rendered in a single-paged table may also be a contributing factor to the delay; tables are rather slow to render at times (#1232).

[Griatch]

As an update to this, I have a prototype EvMore in a local branch which understands pagination and allows you to customize the presentation of each page; so you need to override EvMore with a child class that understands your particular data (building a new EvTable for every page rather than making one huge EvTable and letting EvMore split it afterwards). So a bit more work, but it gives a 100x speedup - listing one page out of 1000 prototypes could take 40s for me, now it takes 0.4s. The remaining issue is still with displaying a mix of module-based and db-based prototypes (mixed data types into EvMore is not supported yet), so more work is needed on that.

[Griatch]

This adds a considerably revamped EvMore; prepared so that one can make a child class to implement custom-display of every page. This is necessary in order to display an EvTable per-page (instead of having a complete EvTable and split that). A custom PrototypeEvMore class now handles both the module-based and db-based prototypes in parallel, shifting the module ones to the end of the listing (presumably the db ones changes more). This allows full pagination while viewing all prototypes together like before (do spawn/list modules like before to only view module-based prototypes).

As said, this gives me speedups of about 100x. I also applied these changes to the scripts listing to properly paginate it.