Do we need the search_without_count() functions?

I’m submitting a …

• bug report
• feature request

Current behavior:

We have many repetitive functions that I believe could be eliminated.

Expected behavior:

Fewer functions required to deliver the equivalent functionality.

Steps to reproduce: Our scrapers have:

• def search(self, query, num_results, qtype=‘’): and
• def search_without_count(self, query): but perhaps we could remove the second and only have the first.

When we do a search() we are not guaranteeing that we will deliver at least num_results results. If the upstream search engine only delivers 5 results then we will only return that many even if the caller has set num_results to 10. If the upstream search engine does not allow us to control how many results we can get then we are not breaking any guarantee if we deliver less than num_results. We are however guaranteeing that we will deliver no more than num_results.

So, why not have a default value for the num_results parameter (def search(self, query, num_results=10, qtype=‘’):) and do away with the without_count functions? Different scrapers could set their default value higher or lower. The final return could ensure that no more than the requested number of urls are returned with return urls[:num_results]

Thoughts?

[bhaveshAn]

Make your own no need to delete

[cclauss]

Three tests on a popular search term like “Fossasia” for each scraper:

1. do not supply a num_results value and test that default value number of results are returned
2. supply num_results=2 and then test that exactly 2 results are returned
3. supply num_results=27 and then test that exactly 27 results are returned (or explain in the test why fewer that 27 will be returned).

Happy hunting!