Building WPF projects is not deterministic with dotnet.exe

• .NET Core version: 5.0.100
• Windows version: 19042.630
• MSBuild .NET Framework version: 16.8.2.56705

Problem description:

Building a WPF .NET Core project with dotnet.exe is not deterministic, but building it with msbuild.exe is deterministic as expected. This issue also breaks incremental builds.

Actual behavior:

Each execution of dotnet.exe build produces assemblies that are binary different.

Expected behavior:

Building a .NET Core project should result in identical output as long as the input did not change. SDK style projects define <Deterministic>true</Deterministic> by default, which is meant to produce identical output each build.

Minimal repro:

1. dotnet new wpf --name deterministic_bug
2. cd deterministic_bug
3. dotnet build -p:OutputPath=bin\dotnet_1\ -p:IntermediateOutputPath=obj\dotnet_1\
4. dotnet build -p:OutputPath=bin\dotnet_2\ -p:IntermediateOutputPath=obj\dotnet_2\
5. msbuild /restore -p:OutputPath=bin\msbuild_1\ -p:IntermediateOutputPath=obj\msbuild_1\
6. msbuild /restore -p:OutputPath=bin\msbuild_2\ -p:IntermediateOutputPath=obj\msbuild_2\

Compare bin\dotnet_1\deterministic_bug.dll to bin\dotnet_2\deterministic_bug.dll

both are different

Compare bin\msbuild_1\deterministic_bug.dll to bin\msbuild_2\deterministic_bug.dll

both are identical

Deeper look

It seems to be a problem with the XAML markup compiler. If you check obj\dotnet_1\deterministic_bug_MarkupCompile.cache and obj\dotnet_2\deterministic_bug_MarkupCompile.cache, you can see that they differ from line 12 to 15. Those lines represent hashes over <Page>-items, references, <Content>-files and cs-files. Check out GenerateCacheForFileList from CompilerState.cs. The hash generation seems to be straightforward, but for some reason the result is not stable and differs everytime in .NET Core.

The same does not reproduce for msbuild.exe. Comparing the content of obj\msbuild_1\ and obj\msbuild_2\ result in identical files.

[MichaeIDietrich]

So, today I had some time to look into it again and I might have finally found the cause of the different output.

It seemed that my first intuition was not that wrong. While it is correct that the Dictionary class keeps track of the insert order, the WPF code often uses its non-generic counterpart HashTable (and HybridDictionary), which do not keep track of the insert order, which results in different entry order between multiple runs when iterating over it.

After doing some debugging I came to the XmlnsCache class, which is more or less responsible for mapping XAML namespaces to assemblies.

The problematic code is where the reference assemblies are being scanned:

private void AddReferencedAssemblies()
{
    if (_assemblyPathTable == null || _assemblyPathTable.Count == 0)
    {
        return;
    }
    List<Assembly> interestingAssemblies = new List<Assembly>();
    // Load all the assemblies into a list.
    foreach(string assemblyName in _assemblyPathTable.Keys) // <-- here
    {
        bool hasCacheInfo = true;
        Assembly assy;
        if (_assemblyHasCacheInfo[assemblyName] != null)
        {
           hasCacheInfo = (bool)_assemblyHasCacheInfo[assemblyName];
        }
        ...

(_assemblyPathTable is here a HybridDictionary)

After patching the foreach loop using a simple OrderBy, I was no longer able to reproduce the issue.

I hope I made no mistake. Maybe someone could verify this by cherry picking this small commit: https://github.com/MichaeIDietrich/wpf/commit/cff984aaf851ba126a822190bf815d0bf5c4d40e

Here again the steps to reproduce, since the steps from my first comment used different output paths which seem to have also an effect on the output for some reason:

1. dotnet new wpf --name deterministic_bug
2. cd deterministic_bug
3. dotnet build
4. Move obj and bin folder to some location outside the project (project is now clean again)
5. dotnet build

Compare the output of both builds.

To test against a local build of the PresentationBuildTasks.dll, one can simply add this line to the csproj file: <_PresentationBuildTasksAssembly>path\to\PresentationBuildTasks.dll</_PresentationBuildTasksAssembly>

The AssemblyReference.cache file is still different, but it does not seem to have a negative effect here.

[ryalanms]

@MichaeIDietrich: Thank you for this. I’ve confirmed your findings. The hash produced is indeed different, and your solution fixes the incremental build issue on WPF for .NET Core. As is, the first comparison against the hashed Reference list is failing every time, causing WPF to always compile.

https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationBuildTasks/MS/Internal/Tasks/IncrementalCompileAnalyzer.cs#L98

PresentationBuildTasks.dll!MS.Internal.Tasks.IncrementalCompileAnalyzer.AnalyzeInputFiles() Line 100 C# PresentationBuildTasks.dll!Microsoft.Build.Tasks.Windows.MarkupCompilePass1.AnalyzeInputsAndSetting() Line 1110 C# PresentationBuildTasks.dll!Microsoft.Build.Tasks.Windows.MarkupCompilePass1.Execute() Line 155 C#

https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationBuildTasks/Microsoft/Build/Tasks/Windows/MarkupCompilePass1.cs#L1094

https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationBuildTasks/MS/Internal/Tasks/CompilerState.cs#L218

Two possible options for next steps are:

1. You can submit a PR with this change and we can review and merge it ASAP.
2. Someone from WPF can submit a fix and we will add you as the coauthor.

Thanks again for the analysis and fix. This is an extremely impactful contribution.

(This will go in to .NET 6.0, but we also want to include this in the next servicing releases for 5.0 and 3.1.)

/cc @dotnet/wpf-developers