question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Caching css_to_xpath()'s recently used patterns to improve efficiency

See original GitHub issue

I profiled the scrapy-bench spider which uses response.css() for extracting information.

The profiling results are here. The function css_to_xpath() takes 5% of the total time.

When response.xpath()(profiling result) was used, the items extracted per second (benchmark result) was higher.

Hence, I’m proposing caching for the recently used patterns, so that the function takes lesser time. I’m working on a prototype for the same and will add the results for it too.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:1
  • Comments:9 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
eliasdornelescommented, Jul 30, 2017

What @lopuhin said! 👍

Though I believe this is the stable backport for functools: https://github.com/MiCHiLU/python-functools32 It would need to conditionally add functools32 to install_requires in setup.py

0reactions
kmikecommented, Jun 29, 2018
Read more comments on GitHub >

github_iconTop Results From Across the Web

Five Data-Loading Patterns To Boost Web Performance
Caching partially stores your data and is not used as permanent storage. Using the cache as permanent storage is an anti-pattern.
Read more >
Caching patterns - Database Caching Strategies Using Redis
Implementing this approach is straightforward and produces immediate performance gains, whether you use an application framework that encapsulates lazy caching ...
Read more >
A Hitchhiker's Guide to Caching Patterns - Hazelcast
Deciding to use caching is just the first step in a long journey. The next step is to think about how your application...
Read more >
Cache-Aside pattern - Azure Architecture Center
Learn how to load data on demand into a cache, which can improve performance and maintain consistency between cached data and the underlying...
Read more >
Common Caching Patterns | Momento Serverless Cache
The first caching choice you need to make is on where to cache your data. In thinking about caching, we often jump to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found