[Feature] Metrics Should Support All-Time Grain Metrics

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

From the beginning of time the metrics issue , people have been asking about non-time based metrics. Some people believed that non-time based metrics were a requirement while others thought that we’d run into practical constraints for usage. George Fraser of Fivetran had this comment:

This isn’t just a technical issue: there are lots of business metrics which are time-series in principle but in practice we choose to calculate at just the present time, because it’s simpler.

He was right! We’ve heard from a lot of our customers that they’re interested in getting single numbers returned for their metrics. The metrics themselves might be time-series in principle but the end consumer is only interested in the present value.

So lets add that functionality.

Implementation

There are two different options we have for adding this type of functionality.

1. Alter the metric node to make timestamp & time_grains optional - if those attributes are not present then we return the single aggregate.
2. Add all_time as an option for the grain input that aggregates across the entire date range.

I am almost certain that option 1 is NOT the route we want to go. Making these parameters optional has the potential to introduce too many issues downstream. For example:

• Our partners rely on their being a grain input. They’d need to change their integrations to support that parameter becoming conditional
• There are a host of validation issues that could arise if we no longer enforce timestamp and time_grains. Metrics that actually do require it wouldn’t know that the metric is misconfigured until they are actually run.

So with that being said, I believe that we should add support for all_time (or some similarly named input) in time grains.

Open Questions

• Should this be defined in the metric like all other time grains or should it be supported in all metrics?
  • I feel strongly that it should have to be defined.
• Should it be called all_time? Is there some other name that better captures it?
• What error message should we raise to show that secondary calculations wouldn’t be supported with this functionality?
  • Should we try and figure out some way of supporting them?

Describe alternatives you’ve considered

Option 1 raised in above message. I think that is a capital B bad idea but I am willing to be convinced otherwise!

Who will this benefit?

Anyone who wants to return metric results across all time.

Are you interested in contributing this feature?

Yep!

Anything else?

No response

[callum-mcdata]

Examples from partners & community members:

• one integration partner wants to return the single metric value in a chrome extension for each metric that would be copied into your clipboard. More of a design example than specific metric type
• Unique customers: one beta customer said they frequently look at the raw count of unique customers and don’t particularly care to chart it over time.

There are a few others like this but tbh we’ve not dug as deep into this use case because it wasn’t supported.

Moving Forward

I think we’re all in agreement that the all_time option is better because it keeps to the current framework and doesn’t throw everything else out. Additionally then customers can look at that time grain and switch to monthly if they want more context.

As for your point @dave-connors-3 around the definition of present_time, that is pretty fair. I think the implementation of that would be all_time as the grain and then some form of parameter passed into end_date - maybe current_date or something like that. all_time is just generic enough that it gives customers the flexibility to specify their time ranges.

[dave-connors-3]

this is an interesting one! Do you have other examples from the community on the kinds of non-time aggregated metrics? the quote above says

there are lots of business metrics which are time-series in principle but in practice we choose to calculate at just the present time, because it’s simpler.

Wouldn’t this imply that the metric is indeed time aggregated, but the way it’s reported depends on a definition of current value or present time? To me, that sounds more like a calendar dimension filter applied to a metric definition than a removal of the time-boundedness of the metric definition itself.

I support the inclusion of the all_time option in any case, but it feels like the examples of single value in my head are like “revenue this quarter” which is more of a final filter view than anything.