Pre-filtering to prevent: too much traffic for live tailing

💡 Feature request

Overview and problem statement

We are seeing exceptions in production that we are struggling to diagnose. We have followed all of the debugging tips here, including wrapping our handler in a try/catch and firing off error logs but I still have no visibility into the errors which suggests to me the exceptions are being thrown outside of the handler. Unfortunately, we don’t know how to replicate these errors and we’re not seeing them in our non-prod environments.

What I would like to do now is pull logs from production, but when trying to use wrangler tail I get the following error.

$ npx wrangler tail --env master | grep "exception" > exceptions.txt
🦚  Setting up log streaming from Worker script "master". Using ports 8080 and 8081.
⠲   This may take a few seconds...
Error: ⚠️  Code 10057: workers.api.error.too_much_traffic_for_live_tailing
🕵️  See documentation

What I’d like to propose is a couple of improvements:

1. Document whether there is any mitigation option available folks who encounter this issue
2. Provide some way to pre-filter logs before they enter the tail pipe

I think many services would seek to provide a sampled tail when traffic is too high, and this might be a great solution, but only if it removed duplicate log entries. If I lost unique (and valuable) exception logs as a result of sampling then that would be equally frustrating.

I understand this request is probably unreasonably difficult, but as it stands today I don’t really have any other options.

Basic example

Provide a --sample flag to indicate that we’re happy to only receive a sample of the logs in the given environment.

$ npx wrangler tail --sample

[bjesus]

has there been any progress on this? we are currently completely unable to tail any of our production workers. Wrangler just shouts Error: ⚠️ Code 10057: workers.api.error.too_much_traffic_for_live_tailing 🕵️ See documentation. As much as the emojis are nice, which documentation exactly?

[nilslice]

We are actively working on a significant improvement to tailing worker logs - we’ll update this thread once there is more to share soon.