Excessive memory usage using endpoint.MapControllerRoute (up to 4GB) for Routes Candidates

Describe the bug

Using the MapControllerRoute function with various route templates the server memory explodes in the order of GB. A project with about 1K of Controllers and each with 30/40 Actions, with a dozen MapControllerRoute endpoints can weight up to 3GB.

We are not using the RouteAttribute decorations, but attributing the routes with MapControllerRoute, for a precise choice and for compatibility with NET.

app.UseEndpoint(endpoint => {
	endpoint.MapControllerRoute({
		name: "RouteName",
		pattern: "api/{controller}/{action}",
	});
	...
	endpoint.MapControllerRoute({
		name: "RouteName",
		pattern: "api/{controller}/{id}/{action}",
	})
	..
});

The memory is almost completely occupied by the Microsoft.AspNetCore.Mvc.ModelBinding.Metadata.DefaultModelMetadata class and the internal routing dictionaries.

If, instead of with the UseEndpoint, the same identical routes are linked with UseMvc, there is not problems, and the application has a normal use of RAM (<500MB in Debug, 150MB in Release)

// this works without leak
app.UseMvc(route => {
	route.MapRoute(
                name: name,
                template: routeTemplate,
                defaults: defaults,
                constraints: constraints);
});

To Reproduce

To replicate the problem I created a test project, with 1k controller with 30 actions, and I defined a series of Route patterns, and the RAM usage is also excessive, reaching 3GB.

https://github.com/Sbaia/WebApp.Core.MemoryUsage

Exceptions (if any)

No Exceptions was thrown

Further technical details

• ASP.NET Core 5 (tested also on 3.1, with same results)
• Visual Studio 2019
• Include the output of dotnet --info

dotnet --info
.NET SDK (che rispecchia un qualsiasi file global.json):
 Version:   5.0.102
 Commit:    71365b4d42

Ambiente di runtime:
 OS Name:     Windows
 OS Version:  10.0.17763
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.102\

Host (useful for support):
  Version: 5.0.2
  Commit:  cb5f173b96

.NET SDKs installed:
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.500 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.1.401 [C:\Program Files\dotnet\sdk]
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.102 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

[NinjaCross]

First of all, thankyou for your answer 😃

In short, routing uses a DFA that avoids backtracking by trading off memory for compute time (it “precomputes” all possible destinations ahead of time).

That’s indeed a great feature, but IMHO the developer should be able to decide if and when to apply or modulate this “precomputation”. There are many factors to consider when balancing the resources (CPU and RAM) allocation in an application, and the fact that a single component of the system decide how to balance itself in an opaque/untunable way is a bit offsetting.

There are strategies to reduce the amount of nodes in the tree, but we believe in general this is not a problem for the majority of apps. Apps with these many routes are not common, and even in these cases the advantage of running matching in linear time is normally worth the memory trade-off.

I appreciate your technical analysis, and I see your general point… but I also think there is a flaw in this last sentence. Maybe it’s just a misunderstanding from my side. If it is so, please accept my apologies in advance 😃

I agree that big applications are not as common as small/average ones, but with this statement you are implicitly saying that it’s ok to build small applications using ASP.NET CORE MVC, and it’s not ok to use it for big applications. It’s an invitation to (average/big) companies (usually creating big products) to move away from ASP.NET CORE MVC and transition to other technologies (… and that’s exactly what my customers are asking me to do right now due to this unsolvable problem). I feel this is a very dangerous statement, and accepting it could cause an unwanted precedence.

[sstraus]

@davidfowl Probably many others had the same problem, especially when porting big old Net Framework applications, and decide to quit instead of discussing it here. So Net Core is just for simple and small applications? I dont think so, but your answer is pointing to a different direction…