Performance Audit attribution from third-party wrapping

Feature request summary

We are a third-party analytics provider that provides Real User Monitoring (RUM) and other features such as JavaScript Error Tracking through a JavaScript library our customers include on their pages called Boomerang.

Lately, we’ve had several customers raise support issues over their concerns that our JavaScript library is causing excessive JavaScript CPU usage, according to Lighthouse/PSI reports.

Here’s an example:

Upon investigation, we found that our Error Tracking plugin was causing Lighthouse to be confused about work being done in our customer’s app. The plugin wraps several high-level functions, and the callbacks of, setTimeout(), addEventListener() and others. It does this to ensure that the customer’s application can get detailed error messages (instead of the generic Script error. message) when errors are generated from scripts in other origins. More details are here, and other error tracking libraries do this wrapping as well.

Because Boomerang is wrapping the callbacks in setTimeout()/etc, any function in any file that calls setTimeout() will be attributed to Boomerang instead, because Boomerang’s wrapped callback is at the “bottom” of the stack. As you can imagine, this accounts for a lot of the JavaScript CPU time on many sites. The actual work done in the wrapped function is minimal (but non-zero), but the time of the entire callback is attributed to Boomerang instead of the “real” work being done. Our understanding is the “URL” of the JavaScript boot-up time /etc tables are based on the function/URL at the bottom of the stack.

If we were to disable Error Tracking and re-load the same page, we can see the real attribution moves a lot to the Page and other scripts (as well as Boomerang), but the “cost” of Boomerang has decreased from 770ms to 127ms:

We were hoping there would be a way to change it so these wrapping functions don’t get blamed for the entire callback cost. One idea would be to have a list of files or functions that could be excluded from “blame” if at the bottom of the stack, similar to library blackboxing.

We’re not sure if that’s feasible though:

• We’d still want the non-wrapping code in Boomerang to be judged based on the work it does, of course.
• We’d like for our customers’s Lighthouse/PSI runs to have this exclusion applied without them having to make any changes or configure any lists (can this be built-into Lighthouse?)
• Each of our customers load our third-party library from distinct URLs

The only other alternates we’ve come up with is better education (explaining the issue to the customer, which we’re doing now), or disabling Boomerang’s wrapping if Lighthouse is detected (which we’d prefer to not do).

I have a repro page here, which merely does a setTimeout() that busy-loops for 2 seconds:

• Boomerang Error Tracking enabled: http://dev.nicj.net/lighthouse-attribution/?errortracking=on

• Boomerang Error Tracking disabled: http://dev.nicj.net/lighthouse-attribution/?errortracking=off

I’m willing to work on this change, if there’s agreement on a correct approach.

What is the motivation or use case for changing this?

To accurately reflect JavaScript CPU cost when third-party libraries are in the callstack.

How is this beneficial to Lighthouse?

More accurate JavaScript CPU attribution.

[patrickhulce]

That is correct, #7059 should solve your customer-facing issue 👍

I think it just needs to be reviewed so no further action needed on your part!

[nicjansma]

@cheneytsai Yeah I certainly wouldn’t expect Boomerang to be anything more than ~200ms in most scenarios.

If you have the trace file, you could share it (or drop me an email nic at nicj dot net) and we can take a look.