Performance Issue: _get_absolute_url() is taking long time with a lot of categories

Issue Summary

When using a new profiling tool PyLive to profile oscar, we find function _get_absolute_url() is taking a lot of time when there are thousands of categories. And time it takes is growing with the number of categories.

Steps to Reproduce

1. Create thousands categories in sandbox
2. Load the page and it will become slower (2-3s to load main page when categories grows to 4000)

Any other relevant information. For example, why do you consider this a bug and what did you expect to happen instead?

The category url does not change over time normally. And it’s intensively used in nearly every product page twice (one in side bar, another in the drop-down list ‘Browse Store’). So it’s naturally that we use cache to cache the url result so we can reuse it over different page.

A simplest way is use Django cache to cache the generated url:

def _get_absolute_url(self, parent_slug=None):
    full_url = cache.get("pk="+str(self.pk))
    if full_url is None:
        full_url = reverse('catalogue:category', kwargs={
                'category_slug': self.get_full_slug(parent_slug=parent_slug),  'pk': self.pk
            })
    cache.set("pk="+str(self.pk), full_url)
    return full_url

When I test it out with PyLive to profile, it can improve the speed by 3x after cache all the urls.

Technical details

• Python version: Python3.6.9
• Django version: 2.2.12
• Oscar version: 2.0 We use PyLive for profiling.

[solarissmoke]

I think Oscar has an appropriate level of caching here - the default category tree in the templates is intended to be used for relatively small number of categories - if you have thousands then I cannot see a situation in which it would make sense to render the whole tree on every view like that - so projects with such structures would be expected to override the applicable templates.

[solarissmoke]

I tend to agree with @rik here. Being too aggressive with caching by default has potential to cause other problems and make debugging them harder.

Also I don’t think get_absolute_url() is the best place to do this caching. If you have a large category tree, you should probably do template caching in the places you render that tree in its entirety, which will be even more efficient than calling get_absolute_url() for thousands of categories (side note - would a tree containing thousands of categories really be usable/desirable to render all at once?). The respective templates in Oscar are easy to override.