LogStats Extension should log the IPM and RPM to the stats on spider_closed signal

Summary

By default, the scrapy.extensions.logstats.LogStats log the details below to the standard logger every 60 seconds:

• IPM (Items per Minute)
• Items scraped
• RPM (Requests per Minute)
• Requests produced

Motivation

We often find the need to retrieve the RPM and IPM after a job is finished. We can calculate this manually or we can check the logs for the last update of LogStats. However, this is tedious and inefficient as LogStats already has this data but isn’t putting them somewhere that is readily available after the job is finished.

Describe alternatives you’ve considered

We can pull out the last update of LogStats from the logs but pulling it from the API and filtering it out is expensive.

Additional context

None

[BurnzZ]

Ahhh I see what you mean now @Gallaecio. Great catch! 🎉 I’ve updated the PR now for this new approach. 😃 Let me know what you think. If all is good with this approach I’ll proceed in writing the tests.

[Gallaecio]

#4111 would log 100 in this case, right? Shouldn’t we instead log the totals divided by the run time (200 in this case)?