Add SignCheck feature to exclude a nupkg from validation if it contains a marker file

• This issue is blocking
• This issue is causing unreasonable pain

This is blocking arcade-powered source-build until either the feature is implemented, or we decide we won’t do it and instead go with an alternate approach like excluding the nupkg from validation using a wildcard in the yaml (https://github.com/dotnet/arcade/issues/6806).

---

The arcade-powered source-build plan involves every repo making an intermediate nupkg. They look like: Microsoft.SourceBuild.Intermediate.{repo-name}.{version}.nupkg they are built on Linux and unsigned, with a source-built unsigned version of every nupkg inside in artifacts/.

I asked @chcosta about how to exclude this from signing validation by default from the Arcade side, and I liked this idea the best:

Add a marker file to the intermediate nuget package that SignCheck would obey. If SignCheck sees the marker at the root of a nupkg, it allows the nupkg and all its contents (recursive) to be unsigned.

This lets us leave SignCheck command lines and the yaml exactly the same. The marker could be an empty .intentionally-not-signed file at the nupkg root, or something weirder or more complicated if there’s concern about this being used inappropriately. Here’s a sample intermediate nupkg with nested nupkg to maybe try it out on later: Microsoft.SourceBuild.Intermediate.sourcelink.1.1.0-ci.zip

---

There are some other alternatives that I think aren’t quite as good, but some don’t require SignCheck changes:

• The eng/SignCheckExclusionsFile.txt is checked into each repo, and I don’t think we should require every repo to have that file just because it participates in source-build.
• We could edit the eng/common template yaml to use an itemPattern that avoids downloading the intermediate nupkgs by filename convention.
  • The filename prefix seems like it could only ever apply to intermediate nupkgs, but there’s still a degree of uncertainty.
  • Makes it harder to repro locally: have to manually filter out PackageArtifacts yourself before running SignCheck.
• Add ExcludeFiles option to SignCheck that accepts wildcards, like InputFiles
  • This is similar to the yaml itemPattern, except it requires a signcheck change. The benefit being that the filtering is a feature of the tool, rather than having to code around the problem in yaml
  • (@chcosta raised this as a possibility. Same issue where a wildcard doesn’t quite seem ideal. However, ExcludeFiles would be included in the SignCheck command, so it wouldn’t be as bad as a yaml change for local reproability.)

---

@chcosta @mmitche @joeloff is the marker a reasonable feature to add?

[joeloff]

Yes and no. The exclusion processing will need a bit of a redesign to better handle this, but it should be feasible.

[dagood]

Ah, I see. Same result here. Would it be reasonable for me to add these lines to the default exclusions right now, to get source-build unblocked?

Exclusions.Add(new Exclusion("Microsoft.SourceBuild.Intermediate.*.nupkg;;Arcade-powered source-build intermediate nupkg"));
Exclusions.Add(new Exclusion("*;Microsoft.SourceBuild.Intermediate.*.nupkg;Arcade-powered source-build intermediate nupkg"));
Exclusions.Add(new Exclusion("*;*.nupkg;Arcade-powered source-build intermediate nupkg"));

I suppose it’s not common to have a nupkg inside a nupkg, so the generality is maybe not too bad. 😄 Then once the relative path bug is fixed, we could make the exclusion more specific.