SingleFile Bundling should deduplicate path conflicts

Description

A recent package update to a third party library broke a project we have that compiled to a single file executable. The error message was the dreaded Error MSB4018: The "GenerateBundle" task failed unexpectedly , Found multiple entries with the same BundleRelativePath error.

Using the nice features released in net6 preview I was able to narrow the file collision down to these two entries - the xml file that’s part of the FSharp.Core library. Frustratingly in this case, an upstream problem causes the FSharp.Core.dll library to be included twice and as of release 5.0.1 for that library, they added an xml file that is now copied twice into the final binary. It doesn’t affect multi file distributions - presumably just copes over, but for single file it stops with this error (as of net6) and a more cryptic error without the paths in net5 and earlier.

System.ArgumentException: Invalid input specification: Found entries 
'C:\xxxxx\.nuget\packages\fsharp.core\5.0.1\contentFiles\any\netstandard2.0\FSharp.Core.xml', 
'C:\xxxxx\.nuget\packages\fsharp.core\5.0.1\contentFiles\any\netstandard2.0\FSharp.Core.xml' with the same BundleRelativePath 'FSharp.Core.xml

I have a attached a tiny project below that reproduces the setup. The duplication comes from one project referring to another (both F#) projects, and using the paket package manager which manages the msbuild instructions. I honestly can’t tell why there are two copies but it’s non-trivial to prevent unfortunately, so it’s not an option to reconfigure the project (I tried for many hours).

I would expect the bundler to deduplicate the source /target paths like the one above and conclude there is no collision. Obviously attempting to bundle two different libraries into the same output file would be bad but this is not the case.

Configuration

• "5.0.200 (also happens on all later net6 versions up to nightly release)
• Windows
• x64
• Appears to be a general issue with the platform
• F# application (for what it’s worth)

Regression?

In this case, it is not a net runtime issue that caused the problem but a library change. Unfortunately this scenario is common enough to be a major headache. Others had reported the same problem with the triggering library by the time I diagnosed the root cause. Short term the library can be altered but it would be strongly preferable for the bundler to simply dedup the list before deciding if there are collisions

Other information

Same reproduction.

Playground.zip

[FSharp library causing the issue]((https://github.com/dotnet/fsharp/issues/11111)

[vitek-karas]

I would very much prefer for SDK to deduplicate this before it gets to single-file. There are potentially other problems if the file is listed twice. We might run crossgen on it twice (if crossgen is enabled), we might pass it to trimmer twice (which is probably OK, but still) and so on.

[sfoslund]

Closing this per https://github.com/dotnet/sdk/issues/16576#issuecomment-815316382