Document process for new custom metrics

Custom metrics are a powerful way to extract insights from pages directly from the JS runtime context. This is an especially useful time to analyze things that depend on the DOM, for example, like counting HTML element and attribute usage. These are things that can be approximated using regular expressions in the response bodies, but it becomes extremely costly considering that the latest desktop and mobile response_bodies tables are over 11 TB combined.

I’d like to use this issue to discuss what the criteria should be for the custom metrics. What are the limitations of doing more work in custom metrics? Should we be actively cleaning up unused custom metrics? Should we have a policy for allowing one-time custom metrics by request?

cc @bkardell

[pmeenan]

Can you combine a bunch of the checks into a single collection script so we’re not running dozens of scripts? It can return JSON and it should be passed through. I’d be surprised if even dozens of metrics added more than a few seconds to the crawl and should be a non-issue but be mindful of any that look particularly expensive (exponential based on size of DOM for example).

On Sat, May 25, 2019 at 4:56 PM Rick Viscomi [e-mail] wrote:

@igrigorik https://github.com/igrigorik @pmeenan https://github.com/pmeenan @paulcalvano https://github.com/paulcalvano I’m looking at the initial list of metrics brainstormed for the Almanac and for many of them I think custom metrics would be useful if not the only practical way. The questions raised in this issue are becoming more important if we’re going to consider adding ~dozens of new custom metrics. The high level question to answer is, how much can we realistically do using custom metrics without affecting the crawl?

One use case that comes to mind is the word count per page listed in the SEO chapter https://github.com/HTTPArchive/almanac.httparchive.org/issues/12#issuecomment-495914982. There are many other metrics with complicated implementations in the rest of the chapters.

(@HTTPArchive/data-analysts https://github.com/orgs/HTTPArchive/teams/data-analysts FYI)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/HTTPArchive/httparchive.org/issues/138?email_source=notifications&email_token=AADMOBKPSGAEKB6BXK2V4BTPXGRYTA5CNFSM4HEOT5S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWHZBMQ#issuecomment-495947954, or mute the thread https://github.com/notifications/unsubscribe-auth/AADMOBO5MOA5ZZGSKWCRP4TPXGRYTANCNFSM4HEOT5SQ .

[rviscomi]

Fixed by https://github.com/HTTPArchive/legacy.httparchive.org/blob/master/custom_metrics/README.md