Have dotnet-sdk-3.0 Linux packages depend on exact dotnet-targeting-pack version

Details here are for RPM packages, but we are in the same situation with Debian packages:

$ repoquery --requires dotnet-sdk-3.0 | grep dotnet-targeting
dotnet-targeting-pack-3.0

This doesn’t express the actual dependency. dotnet-sdk-3.0 contains a file specifying that it needs a specific full major.minor.patch of the targeting pack. This means that e.g. dotnet-sdk-3.0-3.0.101 is compatible with dotnet-targeting-pack-3.0-3.0.0, but patching to dotnet-targeting-pack-3.0-3.0.1 breaks the SDK.

See https://github.com/dotnet/core/issues/3868 for @craigjbass’s report on the way this can break SDK scenarios.

Right now we’re relying on the latest version of dotnet-sdk-3.0 and dotnet-targeting-pack-3.0 on our Linux repos to happen to work with each other. This may also cause people headaches if they try to set up an old SDK version, because RPM doesn’t know how to find the right targeting pack for them.

Mitigations:

• The latest versions should match up and work if our releases go smoothly. This loose dependency is causing additional pain when the repo is messed up.
• The SDK will download targeting packs if they don’t exist. This only works for online build scenarios.

/cc @leecow @omajid @dleeapho

[nguerrera]

does the sdk not understand it can use a newer targeting-pack?

Let’s take a step back here…

A targeting pack consists of reference assemblies, which consist entirely of surface area without implementation. By design and definition, the surface area of the frameworks are tied to TFM (target framework moniker) and we do not include the patch in the TFM. This allows code compiled for a given TFM to work irrespective of patch version of the ultimate runtime environment.

The targeting pack’s version has three parts, like all nuget packages, but you should not interpret the third part as mapping to a specific framework patch number. For example, targeting pack Microsoft.NETCore.App.Ref 6.0.2 would be the second revision to the targeting pack for netcoreapp6.0. It has nothing do with the security fixes in version 6.0.2 of the runtime.

For the most part, revisions to targeting packs should be limited. There are very few things you can revise. (We did revise the ASP.NET Core 3.0 targeting pack to add missing type forwards that should have been there all along. I do not expect this to be the common case at all.)

For any given TFM and shared framework, the SDK is bound to a 3 part (or 3 part with prerelease) specific version of the targeting pack. There are practical reasons for this, such as determinism and performance when pulling targeting pack from nuget. This means that it will not just use 3.0.1 because there is one, it has to be taught to start using 3.0.1. I do not recommend that source build change the TP version and this mapping. It is possible to build 3.0.0 from source either by building at the same commit as the official 3.0.0 pack on nuget.org, or building a newer commit that nevertheless produces 3.0.0, with validation that the content hasn’t changed.

With respect to the mechanics of this, my understanding is that reference assemblies need to be built at the very beginning of the framework build so that they can be consumed by the rest of the build. Furthermore, I heard the idea floating around of pipelining the larger product build to start building downstream repos as soon as reference assemblies are available and defer testing until all of the implementations are available. To give better build throughput. If this were formalized, it seems like it shouldn’t be especially hard to allow for the reference build phase to be performed at a different commit.

[dagood]

To clarify what the Microsoft build is doing, yes, the new SDK depends on the previously built targeting pack bits. Here are the GA and 3.0.1 servicing build outputs:

• 3.0.0 product: 3.0.100 SDK and 3.0.0 netcoreapp targeting pack
• 3.0.1 product: 3.0.101 SDK and 3.0.1 netcoreapp targeting pack
  • This targeting pack should not have been released, because 3.0.101 still strongly depends on 3.0.0 from the previous build.
• In 3.0.2+ we won’t create a targeting pack at all unless it carries a patch (so it can’t be accidentally used or published) https://github.com/dotnet/core-setup/pull/8827

I thought about how source-build works with this in https://github.com/dotnet/core-setup/issues/8735, because yeah, it’s not friendly. What we have now is that Core-Setup will produce the targeting pack, branded as the last-released patch version. I have a small table of Microsoft builds vs. source-build in https://github.com/dotnet/arcade/pull/4318 to demonstrate.

Note the big assumption here: that it’s possible to take e.g. v3.0.5 sources and build a targeting pack with the same contents as v3.0.0, then brand it as 3.0.0. There’s risk of unwanted changes getting in v3.0.5 that won’t be detected by the Microsoft build because it uses old binaries, but change how the source-build product functions.

This is a general problem that I have https://github.com/dotnet/source-build/issues/923 filed for, which applies to more than just targeting packs. Most notably, the CoreFX OOB packages, which aren’t in the netcoreapp shared framework, but could be used by SDK tooling repos and/or the ASP.NET Core shared framework.

Is there some way to verify a set of RPM packages (or a .NET Core SDK installation) has a self-consistent set of sdk-and-targetting-packs even if the machine is online?

What I did is notice that after I dotnet new’d a new project after a fresh install, I suddenly had ~/.nuget/packages/microsoft.netcore.app.ref/3.0.0. 😄 But a more static way post-install would be something like this:

$ for x in /usr/share/dotnet/packs/*; do
    echo $x
    ls $x
    grep -A 1 $(basename $x) /usr/share/dotnet/sdk/3.0.101/Microsoft.NETCoreSdk.BundledVersions.props
    echo
  done

/usr/share/dotnet/packs/Microsoft.AspNetCore.App.Ref
3.0.1
                              TargetingPackName="Microsoft.AspNetCore.App.Ref"
                              TargetingPackVersion="3.0.1"

/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64
3.0.1
# (Note, this one doesn't work because of the RID, searching for Microsoft.NETCore.App.Host gets the expected version.)

/usr/share/dotnet/packs/Microsoft.NETCore.App.Ref
3.0.1
                              TargetingPackName="Microsoft.NETCore.App.Ref"
                              TargetingPackVersion="3.0.0"

/usr/share/dotnet/packs/NETStandard.Library.Ref
2.1.0
                              TargetingPackName="NETStandard.Library.Ref"
                              TargetingPackVersion="2.1.0"

(This is with the bad 3.0.1 netcoreapp targeting pack installed, can see the mismatch.)