`CompileRenderAsync` fails to find `System.Threading.AccessControl` with AI SQL diagnostics enabled in Azure App Service

Describe the bug Calling CompileRenderAsync:

• On embedded resource cshtml files
• In an Azure App Service execution environment
• Targeting netcoreapp2.2 for the web app / netstandard2.0 for a class library with the templates
• With the Application Insights SQL dependency tracking enabled

Results in FileNotFoundExceptions for system reference assemblies being thrown from RazorLight, even with MvcRazorExcludeRefAssembliesFromPublishDeploy turned on. System.Threading.AccessControl is one example, System.CodeDom is another.

To Reproduce Steps to reproduce the behavior:

• Create two projects, one targeting netstandard2.0 (a class library) and a web project targeting netcoreapp2.2
• Embed a template cshtml file in the class library
• Build engine like this (had to do it this way to get unit tests for rendering working):

new RazorLightEngineBuilder()
                .UseEmbeddedResourcesProject(
                    Assembly.GetExecutingAssembly(),
                    rootNamespace: myModelNamespaceRelativeToAssemblyRoot)
                .SetOperatingAssembly(Assembly.GetExecutingAssembly())
                .Build();

• Add something that calls CompileRenderAsync like this:

 Engine.CompileRenderAsync(EmbeddedResourceFilename, myModel);

• Set MvcRazorExcludeRefAssembliesFromPublishDeploy to false in the class library (this allows rendering to work at all with embedded templates and netcoreapp2.2)
• dotnet publish the web project, which references the netstandard2.0 class library
• Deploy the project to an Azure App Service
• Enable the SQL diagnostics option:
• Call your API that renders a razor template. Observe failure. Disable the option, retry the same operation, observe success.

Expected behavior Email renders without exception

Information (please complete the following information):

• OS: Azure App Service, so I suspect Windows Server 2016/2019
• Platform: .NET Core 2.2
• RazorLight version: 2.0-beta1 (latest as of this writing?)
• Visual Studio version: N/A

Additional context I realize it’s a complicated repro, and the maintainer is doing this for free in his free time, but I hope this at least helps someone else debug this issue. I’m not sure what the SQL dependency tracking is doing to cause this issue - maybe dropping assemblies in strange places on the Azure App Service VM?

Unfortinuately, I have not been able to reproduce this issue locally on Windows or Linux functionally or by writing unit tests that I would expect to fail.

The workarounds are to disable SQL tracking, or not target netcoreapp2.2 - I know for a fact targeting net47 instead works fine, but that’s obviously not ideal.

Full stack trace:

System.IO.FileNotFoundException: Could not load file or assembly 'System.Threading.AccessControl, Version=0.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Threading.AccessControl, Version=0.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, IntPtr ptrLoadContextBinder)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, IntPtr ptrLoadContextBinder)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at RazorLight.Compilation.DefaultMetadataReferenceManager.GetReferencedAssemblies(Assembly a, IEnumerable`1 excludedAssemblies, HashSet`1 visitedAssemblies)+MoveNext()
   at RazorLight.Compilation.DefaultMetadataReferenceManager.GetReferencedAssemblies(Assembly a, IEnumerable`1 excludedAssemblies, HashSet`1 visitedAssemblies)+MoveNext()
   at RazorLight.Compilation.DefaultMetadataReferenceManager.GetReferencedAssemblies(Assembly a, IEnumerable`1 excludedAssemblies, HashSet`1 visitedAssemblies)+MoveNext()
   at RazorLight.Compilation.DefaultMetadataReferenceManager.GetReferencedAssemblies(Assembly a, IEnumerable`1 excludedAssemblies, HashSet`1 visitedAssemblies)+MoveNext()
   at RazorLight.Compilation.DefaultMetadataReferenceManager.GetReferencedAssemblies(Assembly a, IEnumerable`1 excludedAssemblies, HashSet`1 visitedAssemblies)+MoveNext()
   at System.Linq.Set`1.UnionWith(IEnumerable`1 other)
   at System.Linq.Enumerable.UnionIterator`1.FillSet()
   at System.Linq.Enumerable.UnionIterator`1.ToArray()
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at RazorLight.Compilation.DefaultMetadataReferenceManager.Resolve(Assembly assembly, DependencyContext dependencyContext)
   at RazorLight.Compilation.DefaultMetadataReferenceManager.Resolve(Assembly assembly)
   at RazorLight.Compilation.RoslynCompilationService.EnsureOptions()
   at RazorLight.Compilation.RoslynCompilationService.get_ParseOptions()
   at RazorLight.Compilation.RoslynCompilationService.CreateSyntaxTree(SourceText sourceText)
   at RazorLight.Compilation.RoslynCompilationService.CreateCompilation(String compilationContent, String assemblyName)
   at RazorLight.Compilation.RoslynCompilationService.CompileAndEmit(IGeneratedRazorTemplate razorTemplate)
   at RazorLight.Compilation.RazorTemplateCompiler.CompileAndEmit(RazorLightProjectItem projectItem)
   at RazorLight.Compilation.RazorTemplateCompiler.OnCacheMissAsync(String templateKey)
--- End of stack trace from previous location where exception was thrown ---
   at RazorLight.EngineHandler.CompileTemplateAsync(String key)
   at RazorLight.EngineHandler.CompileRenderAsync[T](String key, T model, ExpandoObject viewBag)

[jzabroski]

@tghamm It’s important to realize this is not our bug. This belongs with either dotnet org or with Azure App Service. The way you can troubleshoot this problem to help get Microsoft to move the ball on this problem is as follows:

1. Read https://docs.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext
2. Use Type.GetType("TypeNameThatIsFailedToBeFound").Assembly to log the assembly location of the type as the .NET Core Runtime thinks it exists, vs. the one that the particular call graph wants loaded
3. Use the tricks explained in https://docs.microsoft.com/en-us/dotnet/core/dependency-loading/understanding-assemblyloadcontext#debugging-type-conversion-issues to layer on top of Type.GetType mentioned above, by running: System.Runtime.Loader.AssemblyLoadContext.GetLoadContext(Type.GetType("TypeNameThatIsFailedToBeFound").Assembly)
4. Log your host configured probing paths: https://docs.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#host-configured-probing-properties
5. Note also ADDITIONAL_DEPS https://docs.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#how-are-the-properties-populated - the article doesn’t mention it but all deps.json files can be retrieved also from System.AppContext.GetData(“APP_CONTEXT_DEPS_FILES”), so you should log those as well.

[jzabroski]

Hi Kevin, Thanks, that is basically what I’m interested in hearing. I think rather than me spend time debugging each person’s individual set-up, it makes more sense for me to define some best practices samples people can fork off of, and then ask them, ok, which sample did you base your logic on? And you captured beautifully what you want to achieve - unit test code coverage, and deploy via web app.