Upgrading Visual Studio to 16.4 broke the execution of an In-Process asp.net core 3.0.1 application. Error 500.30 ANCM

Describe the bug

Just after upgrading to VS 2019 16.4, i cannot execute (CTRL+F5) one of my asp.net core 3.0 projects (3.0.1 actually) (using IISExpress). I did not upgrade the project to 3.1.

Other solutions on 500.30 error did not work for me

Update 1

Executing with dotnet run, succesfully starts up the application at localhost:5000

Update 2

When changed the settings in the debug section of the project, I chose IISExpress and out of process model, it executes succesfully with CTRL+F5

Update 3

After fiddling around I came up with these observations:

1. I build the application in my local pc. In my pc there is installed .net core SDK 3.1. I publish to a local folder, and then I xcopy the files to the actual server were the site is hosted under IIS. In the server there is installed only the .net core 3.0 run-time. The site fails to run on the server with the same 500.30 ANCM error.
2. I edit the web config on the site on the server and switch to “OutOfProcess”. No other change. The binaries are the same that failed before. This time the site starts up successfully.
3. I put a global.json file in my solution folder and specify SDK version 3.0.100. I built again, publish to local folder and xcopy to server. First observation is that this time the total size of files to xcopy is different even though both times I targeted .netcoreapp3.0. I would expect that the size is the same. Changing the SDK to use, changes the actual assemblies to include or reference??
4. On the server, I switch back to InProcess in web.config. Now I have the assemblies published with the 3.0.100 SDK from my pc. The site starts successfully even InProcess.

So when I build with latest SDK (3.1) the site fails to start InProcess even in a machine with only 3.0 run-time installed. When I build with specified SDK 3.0.100 the site starts successfully on server machine.

In every case, the project is netcoreapp3.0, not 3.1

Initial error output

Output in browser:

HTTP Error 500.30 - ANCM In-Process Start Failure
Common causes of this issue:
The application failed to start
The application started but then stopped
The application started but threw an exception during startup
Troubleshooting steps:
Check the system event log for error messages
Enable logging the application process' stdout messages
Attach a debugger to the application process and inspect
For more information visit: https://go.microsoft.com/fwlink/?LinkID=2028265

When debugging (F5) the application executes the ConfigureServices method, but it doesn’t enter the Configure method

I have some swashbuckle nuget packages referenced in rc versions. I thought they were the problem but even when I removed all references the project still did not start.

Executing the exact same codebase from another pc with vs 2019 16.3.10 installed did not have any problems.

The only exception I could see from Debug console is this: Exception thrown: ‘System.NullReferenceException’ in System.Private.CoreLib.dll

This exception did not happen in the vs16.3 execution.

Uninstalling the 3.1 hosting bundle did not solve the problem. The application is supposed to use 3.0.1 right?

To Reproduce

I’m not sure what is needed to reproduce. Other 3.0.1 projects run fine.

Further technical details

dotnet --info:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.100
 Commit:    cd82f021f4

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.100\

Host (useful for support):
  Version: 3.1.0
  Commit:  157910edee

.NET Core SDKs installed:
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Output from debug when I Debug (F5) the project (only the last lines)

'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Text.Encodings.Web.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Reflection.Emit.Lightweight.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\3.0.1\Microsoft.Extensions.Localization.Abstractions.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Data.Common.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Console.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Threading.Thread.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Microsoft.Extensions.Hosting.Internal.Host: Debug: Hosting starting
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\Microsoft.Win32.Registry.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Xml.XDocument.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Private.Xml.Linq.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.0.1\System.Private.Xml.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager: Information: User profile is available. Using 'C:\Users\thanasis\AppData\Local\ASP.NET\DataProtection-Keys' as key repository and Windows DPAPI to encrypt keys at rest.
The thread 0x2fcc has exited with code 0 (0x0).
Exception thrown: 'System.NullReferenceException' in System.Private.CoreLib.dll
The thread 0x95c has exited with code 0 (0x0).
The program '[14572] iisexpress.exe' has exited with code -1 (0xffffffff).

Visual Studio 2019 16.4

Project .csproj (almost)

<Project Sdk="Microsoft.NET.Sdk.Web">
...
	<PropertyGroup>
		<TargetFramework>netcoreapp3.0</TargetFramework>
		<AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
		<AspNetCoreModuleName>AspNetCoreModuleV2</AspNetCoreModuleName>
		<PublishTrimmed>false</PublishTrimmed>
		<PublishSingleFile>false</PublishSingleFile>
		<Nullable>enable</Nullable>
	</PropertyGroup>

	<PropertyGroup>
		<GenerateSerializationAssemblies>Off</GenerateSerializationAssemblies>
		<GenerateDocumentationFile>true</GenerateDocumentationFile>
		<NoWarn>$(NoWarn);1591</NoWarn>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="3.0.0" />
		<PackageReference Include="Microsoft.AspNetCore.Authorization" Version="3.0.0" />
		<PackageReference Include="Microsoft.AspNetCore.Hosting" Version="2.2.7" />
		<PackageReference Include="Microsoft.AspNetCore.Server.IIS" Version="2.2.6" />
		<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="3.0.1" />
		<PackageReference Include="Microsoft.Extensions.Hosting" Version="3.0.1" />
		<PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="3.0.0" />
		<PackageReference Include="Swashbuckle.AspNetCore" Version="5.0.0-rc4" />
		<PackageReference Include="Swashbuckle.AspNetCore.Annotations" Version="5.0.0-rc4" />
		<PackageReference Include="Swashbuckle.AspNetCore.Filters" Version="5.0.0-rc8" />
	</ItemGroup>

	...

</Project>

[analogrelay]

Remove all the package reference versions below 3.0 from the project and clean and re-build the solution. This worked for me!

This is what I would expect to work. Using 2.x versions of Microsoft.Extensions.* and Microsoft.AspNetCore.* packages in an ASP.NET Core 3.0/3.1 project is *not supported.

Yeah I know, let you down on this. Will work on it some more when I have a chance

No worries! We’ll be here when you get something together. We’re very aware that other things can get in the way of setting up a repro, it’s totally fine!

[saysmaster]

Hello. Today I had the time to look at the project again.

Indeed, I can confirm that, in my case the problem was those two Nuget packages, as @jkotalik mentioned:

<PackageReference Include="Microsoft.AspNetCore.Hosting" Version="2.2.7" />
<PackageReference Include="Microsoft.AspNetCore.Server.IIS" Version="2.2.6" />

Removing them I could start the 3.0 project with the 3.1 SDK and InProcess. I don’t even remember how did those two packages end up there in my csproj. Other 3.0 projects in my solution don’t reference those packages.

Thank you @jkotalik