Add support for advanced usage using the controller pattern

Split from #116

POC branch: https://github.com/tusdotnet/tusdotnet/tree/POC/controller-pattern

This issue needs refinement. See earlier discussions in #116 . When implementing this issue we also need to add support for dependency injection, i.e. services.AddTus().

[SnGmng]

First of all, thanks for your feedback 😃 I agreed with many of your suggestions and implemented them in my last commits to the POC

Addressing your comments

In the above setup the TusStorageService always contains the store (storage proxy probably) to use so there is no need for the endpoint to care. Compare this to how you would setup a EF context or a Redis connection. This would also solve the TODO in public virtual Task<ControllerCapabilities> GetCapabilities().

Agreed. I implemented this in my last commits

Since the TusStorageService is “complete” there is no need for the [TusInheritCapabilities(typeof(TusDiskStore))] attribute.

Agreed and removed 😃

I think that the TusEnableExtensionAttribute and TusDisableExtensionAttribute are a good idea but not implemented correctly. These will only not announce the functionality but the functionality will still be there.

Being able to announce extensions independently from TusStore was the reason why i intruduced this feature in the first place. For example, if you want to disable the termination extension but want to keep using TusDiskStore, how would you do it? Just failing all termination requests while still announcing termination extension doesnt really seem like a good idea to me.

I’m not really sure what the usage is for the [TusController] attribute? It seems redundant as the controller must inherit from TusControllerBase anyway. We could probably rename TusControllerBase to just TusController.

The intended usage of [TusController] attribute is for assembly scanning to be able to tell base-classes and implementations apart and for users to disable a controller by removing the attribute. For example if a user creates his own abstract class from TusControllerBase, assembly scanning is able to detect and ignore it, because the [TusController] attribute is missing. Btw i named it TusControllerBase to match the naming convention of MVC’s ControllerBase, but i dont mind to change that, if you dont like it.

Speaking of TusController. I’m not a fan of the abstract methods forcing users to implement each and every method to use this pattern. I can imagine that some users might want to just override a specific method, e.g. they want to add the file id and the current user id to a DB table and thus just wants to override the Create method.

I completely agree 😃 I’ve made the TusController methods virtual in my latest commits

Given the above, we could probably change how we inject the StorageService to be the same as e.g. HttpContext (property injection).

Agreed

I think this comment is interesting but I don’t have all the details. Care to elaborate? 😃 // TODO: // 1. Seperate request validation from file storage validation // 2. Validate request here // 3. Validate file storage in StorageService

The idea is to seperate validation into two parts.

• One for checking if the request is not malformed (done by IntentHanlers)
• One for checking if the operation itself to the storage is valid (done by StorageService)

Here is an example: The Requirement new UploadOffset() checks if a request is malformed, therefore it should be executed in IntentHandler. The Requirement new FileExist() checks if a storage operation is invalid, therefore it should be executed in StorageClient.

This has following advantages:

• The IntentHandlers no longer depend on ITusStore
• If needed, the user can handle invalid storage operations in the controller, by catching its exceptions

Also a thought: Should we integrate with response formatting in ASP.NET Core to allow other responses than simple text/plain? 🤔

I think it could indeed be useful for some users who want to return some json when the upload is finshed for example. I think we can discuss it at a later time

Another idea based on the above: Could we maybe not use our own controller at all but rather integrate the storage service, intent etc into a regular controller as injected services? If so we would not need to implement all this custom code at all.

I think it would be pretty difficult tu ensure it is compliant with tus spec. For example ho can we prevent an user from returning the wrong IActionResult or ensure all headers have been set correctly? Also, a regular controller has a 1:1 request to action mapping, which makes it difficult to implement CreationWithUpload or Concatenation.

Given how much traction “Minimal APIs” in .NET6 is gaining I think that we should pause this feature to not do to much job before we know how Minimal APIs are received by the community. If it turns out that Minimal APIs is the new black and MVC is “legacy” it seems odd to rebuild the entire code base to something that is legacy.

I dont see minimal apis as an replacement for controllers, but rather as an addition to simplify the creation of small applications. Also this poc already kind of supports minimal apis:

var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddTus()
    .AddStorage("my-storage", new TusDiskStore(@"C:\tusfiles\"), true)
    .AddEndpointServices();

var app = builder.Build();

app.MapTus("/files");

app.Run();

Note: In one of the next sections i explain in depth why i think that the controller pattern is still a good idea.

Current design

Configuration

services
    .AddTus()
    .AddController<MyTusController>()
    .AddStorage("my-storage", new TusDiskStore(@"C:\tusfiles\"), isDefault: true)
    .AddEndpointServices();

Storage

I decided to implement a kind of “named clients” pattern to access storage

To register your storage:

services.AddStorage("my-storage", new TusDiskStore(@"C:\tusfiles\"), isDefault: true)

The isDefault: true means, that evey tus controller uses this storage by default, if nothing else is configured. Thats it, now you can use it in a controller:

[TusController]
[TusStorageProfile("my-storage")] // to change the used storage
public class MyTusController : TusControllerBase
{
    public override async Task<ICompletedResult> FileCompleted(FileCompletedContext context)
    {
        // StorageClient has been automatically injected
        await StorageClient.Delete(context.FileId);

        return BadRequest("Upload failed successfully. Please try again :)");
    }
}

To access your storage in DI services, inject ITusStorageClientProvider and then do:

var storageClient = await _storageClientProvider.Get("my-storage");
// or
var storageClient = await _storageClientProvider.Default();

Controllers

Registering a controller is still as simple as doing

endpoints.MapTusController<MyTusController>("/files");

You can configure more options by doing:

endpoints.MapTusController<MyTusController>("/controller/files", options => 
{
    options.StorageProfile = "my-storage";
    options.MaxAllowedUploadSizeInBytes = 1_000_000_000;
    options.MetadataParsingStrategy = MetadataParsingStrategy.AllowEmptyValues;
});

Alternatively you can configure the options using attributes:

[TusStorageProfile("my-storage")]
[TusMaxUploadSize(1_000_000_000)]
[TusMetadataParsing(MetadataParsingStrategy.AllowEmptyValues)]
public class MyTusController : TusControllerBase

Controller alternative

If users dont want to write a controller, they can use this simpler abstraction: (which uses EventsBasedTusController)

// uses the EventsBasedTusController under the hood
endpoints.MapTus("/files", (options) =>
{
    options.StorageProfile = "my-storage";
    options.Expiration = new AbsoluteExpiration(TimeSpan.FromMinutes(Constants.FileExpirationInMinutes));
    options.MetadataParsingStrategy = MetadataParsingStrategy.AllowEmptyValues;
    options.Events = new Events
    {
        OnFileCompleteAsync = ctx =>
        {
             logger.LogInformation($"Upload of {ctx.FileId} completed using {ctx.Store.GetType().FullName}");
             return Task.CompletedTask;
        }
    };
});

Examples of tus controllers

Simplest Tus Controller

[TusController]
public class VerySimpleTusController : TusControllerBase
{
    public override async Task<ICompletedResult> FileCompleted(FileCompletedContext context)
    {
        var tusFile = await StorageClient.Get(context.FileId);

        DoSomeProcessing(tusFile);

        return FileCompletedOk();
    }
}

Using database in a tus controller

[TusController]
public class DatabaseTusController : TusControllerBase
{
    private readonly ApplicationDbContext _dbContext;

    public SimpleDatabaseTusController(ApplicationDbContext dbContext)
    {
        _dbContext = dbContext;
    }

    public override async Task<ICreateResult> Create(CreateContext context)
    {
        var result = await StorageClient.Create(context);

        _dbContext.Insert(result.FileId);

        return CreateOk(result);
    }

    public override async Task<IDeleteResult> Delete(DeleteContext context)
    {
        await StorageClient.Delete(context.FileId);

        _dbContext.Delete(context.FileId);

        return DeleteOk();     
    }  
}

Why controller pattern?

In this section i want to explain why i advocate for a controller pattern and change the current design. This may sound overly negative about the current design which is not my intent 😃 I created the following diagram that will help me illustrate what i mean. (correct me if i’m wrong)

I see the following problems with this design:

• Intent Handlers have both protocol logic and storage logic (for example file expiration)

  This should be seperated to increase modularity.

• User Application Logic is not part of the Request Flow (left to right). Instead, it is a outside observer, only communicating through Events, Callbacks and special classes like IFileIdProvider.

  This adds unneccessary complexity. Making User Application Logic a part of the Request Flow will fix this while giving the user code additional control.

• One big options class for everything, regardless of the scope, is subobtimal.

  Every scope should have their own options. Options should be configured once and not depend on some callback.

• Everything uses ITusStore

  In many classes there is a dependency on TusStore, that isnt really needed. Validation, Configuration and even the Intent Handlers and Protocol Handlers depend on TusStore.

Here is how the new design solves this:

Here is the diagram of the tus controller architecture used in my commit:

• User code is now part of the request flow (left to right)

  Removes the unnesecary need for events. Allows the users to intercept requests and change default behaiviur easily, without having to rewrite TusStore code and open unneccesary issues. Gives user appplication code more power

• Options classes have been split up and moved to different scopes

  Removes the unnesecary need for option callbacks and simplifies configuration

• IntentHandlers, Protocol Handler and Request Validation do not depend on TusStore anymore

  Increases Modularity and users can now implement new features using TusController without needing changes to the TusDiskStore

• Seperation of concerns

  IntentHandlers+Validation have been seperated from storage logic:
  • IntentHandlers now only handle protocol logic and do request validation
  • StorageService now handles storage logic and does storage validation

(Note: In my commit, IntentHandlers are named RequestHanlders and StorageService is named StorageClient)

What are your thoughts about this?

[promontis]

@smatsson Any update on this?