question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Larger filings hold back async from sending close to 10 requests per second

See original GitHub issue

Describe the bug When trying to save a larger filing type (e.g. 10-K), only about 5 filings are being saved per second. This seems to suggest that maybe 10 requests aren’t being sent per second. This seems to be related to the problem we were running into earlier, where there was some sort of latency between requests and the EDGAR server. Not sure if this is fixable or if it is a limitation out of this package’s control.

To Reproduce

from secedgar.filings import MasterFilings

m = MasterFilings(year=2020, quarter=4, entry_filter=lambda e: e.form_type == "10-K")
m.save('temp')

Expected behavior Iterations per second should be closer to 10.

Additional context I know @reteps and I had discussed this earlier, but wanted to mention this in public in case anybody has a good solution or insight into why this problem is coming up.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
retepscommented, Jan 8, 2021

There is no reason it should be 5 it/sec. i was getting 10 on my own.

0reactions
rahul18crackercommented, Jan 8, 2021

@jackmoody11 : Thanks for the clarification, i dint notice this was a bulk call

Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling long Web Requests with Asynchronous Request ...
Figure 2 - Once the request has been submitted, the browser 'pings' the server every few seconds for progress information. In this case,...
Read more >
Asynchronous Requests with Python requests - Stack Overflow
To do multiple tasks with async.map asynchronously you have to: Define a function for what you want to do with each object (your...
Read more >
ASP.NET Core Performance Best Practices | Microsoft Learn
Asynchronous APIs allow a small pool of threads to handle thousands of concurrent requests by not waiting on blocking calls. Rather than waiting ......
Read more >
When to Use (and Not to Use) Asynchronous Programming
Imagine that it takes 10 seconds to get data from the second server. While you are waiting, the whole script is actually doing...
Read more >
New AWS Lambda controls for stream processing and ...
Today AWS Lambda is introducing new controls for asynchronous and stream processing invocations. These new features allow you to customize ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found