Add IOutputCacheBufferWriterStore

Ideally for preview 6…

Background and Motivation

We wish to implement an official SE.Redis output cache implementation.

The existing IOutputCacheStore API is byte[]-focused, which is fine for the in-process implementation which constantly returns the same byte[], but is hugely suboptimal for out-of-process implementations, as we would have to return a right-sized byte[] every call (in either direction), causing GC load. Additionally, since output-cache is an entire page load plus headers, this can easily be large enough to spill into LOH.

Context: https://github.com/dotnet/aspnetcore/pull/48450

For this, we propose three things:

1. a new SetAsync method similar to the existing, but taking ReadOnlySequence<byte> instead of byte[]; this API to be implemented in the existing IOutputCacheStore interface as a DIM, to avoid back-compat problems
2. a new interface IOutputCacheBufferWriterStore which extends IOutputCacheStore (same assembly/namespace), allowing implementations to optionally use a new GetAsync API that takes an IBufferWriter<byte> - thus allowing implementations to pass data to the consumer while site-stepping the topic of “data lifetime” : the receiver can pass a recycling-enabled buffer-writer, and the cache implementation does not need to know about those details - it just asks for space to write data, and writes
3. a new API to register the SE.Redis output cache implementation, in the existing Microsoft.Extensions.Caching.StackExchangeRedis OOB package

Proposed API

  namespace Microsoft.AspNetCore.OutputCaching;

  public interface IOutputCacheStore
  {
      // pre-existing
      ValueTask SetAsync(string key, byte[] value, string[]? tags, TimeSpan validFor, CancellationToken cancellationToken);

    
+     ValueTask SetAsync(string key, ReadOnlySequence<byte> value, ReadOnlyMemory<string> tags, TimeSpan validFor, CancellationToken cancellationToken)
+     {
+         // compatibility implementation using the original API
+         return SetAsync(key, value.ToArray(), tags.IsEmpty ? null : tags.ToArray(), validFor, cancellationToken);
+     }
  }

+ public interface IOutputCacheBufferWriterStore : IOutputCacheStore
+ {
+     ValueTask<bool> GetAsync(string key, IBufferWriter<byte> destination, CancellationToken cancellationToken);
+ }

This second interface is optional / separate because not all types will wish to opt in; in particular, in the case of the existing in-proc implementation, it will be preferable to just keep returning the existing byte[]; as such, the output cache middleware will test for this API and use the most appropriate.

Additionally, to implement the SE.Redis cache, we propose the addition of

package: Microsoft.Extensions.Caching.StackExchangeRedis (pre-existing) type StackExchangeRedisCacheServiceCollectionExtensions (pre-existing)

+ public static IServiceCollection AddStackExchangeRedisOutputCache(
+     this IServiceCollection services, Action<RedisCacheOptions> setupAction)

which is the new API to register SE.Redis into output cache

Alternatives considered

1. eating the GC overhead, zero API change - hugely undesirable in this context
2. have a new GetAsync API that returns some kind of “data with lifetime” - however, there’s no good metaphor for this, and it pushes the buffer management into each individual buffer implementation, rather than having one buffer implementation and having the cache implementations just worry about cache

[halter73]

API Review Notes:

• The issue is that the byte[] buffers are suboptimal for out-of-process performance.
• Might we need backpressure for the IBufferWriter<byte> destination?
  • The default output cache limit 64 MB per response and 100 MB total
  • A PipeWriter would allow us to do smaller writes and chunk it up awaiting backpressure.
• On the flip side, would we ever start writing to the cache before all the data is buffered in memory?
  • If we did, we could pass the PipeReader for the request body, but we want to validate the entire body before we start caching anyway.
• And async TryGet is unusual. Are there any other examples of this in the framework? There’s no out, but it seems to align.
• Does StackExchangeRedisCacheServiceCollectionExtensions make sense for AddStackExchangeRedisOutputCache?
  • Yes. This is the place AddStackExchangeRedisCache which adds IDistributedCache lives.
• Does it make sense for IOutputCacheBufferStore to derive from IOutputCacheStore?
  • Yes. We want the EvictByTagAsync method, and it’s easy to implement the byte[] methods.
  • We do want real implementations for the IOutputCacheStore byte[] methods and not something that just throws then.
• Do we like the IOutputCacheBufferStore?
  • There’s no IBufferWriter anymore but the pipe still gives you buffers.
  • We hear no better proposals.

API Approved!

// Microsoft.AspNetCore.OutputCaching.dll
namespace Microsoft.AspNetCore.OutputCaching;

+ public interface IOutputCacheBufferStore : IOutputCacheStore
+ {
+     ValueTask<bool> TryGetAsync(string key, PipeWriter destination, CancellationToken cancellationToken);
+     ValueTask SetAsync(string key, ReadOnlySequence<byte> value, ReadOnlyMemory<string> tags, TimeSpan validFor, CancellationToken cancellationToken);
+ }

// Microsoft.Extensions.Caching.StackExchangeRedis.dll
namespace Microsoft.Extensions.Caching.StackExchangeRedis;

public static class StackExchangeRedisCacheServiceCollectionExtensions
{
+     public static IServiceCollection AddStackExchangeRedisOutputCache(
+         this IServiceCollection services, Action<RedisCacheOptions> setupAction)
}

[mgravell]

@mitchdenny TryGetAsync?

+ public interface IOutputCacheBufferStore : IOutputCacheStore
+ {
+     ValueTask<bool> TryGetAsync(string key, IBufferWriter<byte> destination, CancellationToken cancellationToken);
+     ValueTask SetAsync(string key, ReadOnlySequence<byte> value, ReadOnlyMemory<string> tags, TimeSpan validFor, CancellationToken cancellationToken);
+ }