AzureWebHookStore.QueryWebHooksAcrossAllUsersAsync downloads entire table into memory

From reading the code it seems this library downloads the entire content of the Azure Table to memory in order to query existing web hooks for ones which match specific actions. Is that correct?

var query = new TableQuery();
var entities = await _manager.ExecuteQueryAsync(table, query);

Has this been tested at scale i.e. is this used by any service with 1000+ active web hooks? As the table content grows, downloading and deserializing the entire table in memory can get pretty slow and eat up a lot of memory. There are also $ costs associated with table operations and network bandwidth at scale. Has that factor been taken into consideration?

Thanks, just trying to understand.

[dougbu]

For your scenario, I’d recommend subclassing AzureWebHookStore and changing the application to use your subclass.

[stefann42]

It’s not going to grow that slowly if you introduce this feature in an existing SaaS product with existing customers who have been asking for it, which is what I’m trying to do 😃. Imagine if an existing Microsoft product adopts this library. The first thing they’d do is change the schema of this storage provider.

The rate of growth isn’t really a relevant point unless you guys designed this library only for devs who are prototyping or building MVPs. If I know I’ll have 1000+ active webhooks it doesn’t make me feel better that I’ll hit that number in say 12 months vs. 3 months. Adopting new schema with scalability issues means you’re signing up for a major schema change at some point in the future. It’s preferable to do it right from the start (unless, of course, you are building a prototype/MVP).