Larger filings hold back async from sending close to 10 requests per second
See original GitHub issueDescribe the bug When trying to save a larger filing type (e.g. 10-K), only about 5 filings are being saved per second. This seems to suggest that maybe 10 requests aren’t being sent per second. This seems to be related to the problem we were running into earlier, where there was some sort of latency between requests and the EDGAR server. Not sure if this is fixable or if it is a limitation out of this package’s control.
To Reproduce
from secedgar.filings import MasterFilings
m = MasterFilings(year=2020, quarter=4, entry_filter=lambda e: e.form_type == "10-K")
m.save('temp')
Expected behavior Iterations per second should be closer to 10.
Additional context I know @reteps and I had discussed this earlier, but wanted to mention this in public in case anybody has a good solution or insight into why this problem is coming up.
Issue Analytics
- State:
- Created 3 years ago
- Comments:8 (6 by maintainers)
Top Results From Across the Web
Handling long Web Requests with Asynchronous Request ...
Figure 2 - Once the request has been submitted, the browser 'pings' the server every few seconds for progress information. In this case,...
Read more >Asynchronous Requests with Python requests - Stack Overflow
To do multiple tasks with async.map asynchronously you have to: Define a function for what you want to do with each object (your...
Read more >ASP.NET Core Performance Best Practices | Microsoft Learn
Asynchronous APIs allow a small pool of threads to handle thousands of concurrent requests by not waiting on blocking calls. Rather than waiting ......
Read more >When to Use (and Not to Use) Asynchronous Programming
Imagine that it takes 10 seconds to get data from the second server. While you are waiting, the whole script is actually doing...
Read more >New AWS Lambda controls for stream processing and ...
Today AWS Lambda is introducing new controls for asynchronous and stream processing invocations. These new features allow you to customize ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
There is no reason it should be 5 it/sec. i was getting 10 on my own.
@jackmoody11 : Thanks for the clarification, i dint notice this was a bulk call