Performance improvement: sfaira.data.store.io_data.read_dao

The sfaira.data.store.io_data.read_dao function spends most of it’s time reading in a pickle file: https://github.com/theislab/sfaira/blob/aeaa60ff128046b7564aa9cccd6293a9300e5a31/sfaira/data/store/io_dao.py#L117

Maybe there is a more efficient way to read in the necessary data.

Here’s a screenshot of the profiler output:

[felix0097]

Just checked - the profiler output does not really make much sense cause the uns data files are super small, so should be fast to read in.

Ran pyinstrument profiler instead of the cProfile and things look much more sensible now:

Looking at this, it doesn’t make much sense to change the current implementation away from using pickle.

[felix0097]

This is fixed now with the above mentioned pull request. Issue was that the Anndata OverloadedDict saved lots of unnecessary data which made the saved pickle files extremely large.