New parameter ExcludeAssembliesWithoutSources to control automatic assembly exclusion

Problem

Coverlet automatically excludes third-party assemblies from the coverage analysis, since they are usually not interesting to the user. The heuristic for determining if an assembly is a third-party dependency is if any of its source documents does not have a corresponding source file.

(Ref: https://github.com/coverlet-coverage/coverlet/blob/3ae19761876f2ac809c227ca3563495e28993b59/src/coverlet.core/Helpers/InstrumentationHelper.cs#L125-L129)

C# source generators and some F# constructs add source documents to firsrt-party assemblies, causing them to be excluded by this heuristic, and compiler bugs may also trigger this. See e.g. #1084 #1145 #1100 #1297

The C# source generators are handled by identifying them using a best practice that the document names should end in “.g.cs”, and similar logic can be added for the other issues. But the source generator naming is but a best practice and not all projects will follow it, and new compiler issues will likely turn up in the future, so this heuristic will continue to cause issues.

Suggested solution

Add a new parameter ExcludeAssembliesWithoutSources with the values MissingAny, MissingAll and None. MissingAny is the current behaviour, but MissingAll would be the new default behaviour to avoid these and future issues with generators. None would be a new setting to never filter on source files, for those rare cases where it may not make sense. Explicit include/exclude settings would override any filtering from this parameter.

Changing the default to include an assembly in the coverage if at least one source document exist may cause some third-party documents to be included, if the same file name exists in the project being analysed with coverlet (pointed out by @daveMueller). Users affected by this can then either change the setting to MissingAny to revert to the old behaviour, or explicitly exclude those assemblies.

Coverlet should still have a heuristic to ignore generated source files according to conventions, but now to avoid logging those cases.

(Edit: change parameter name in feature description to ExcludeAssembliesWithoutSources)

[TomasMorton]

Ran into this because we heavily use this library: https://github.com/mcintyre321/OneOf with the source generator. It generates GenerateOneOfAttribute.cs in all of our projects. This causes coverlet to fail: TpTrace Verbose: 0 : 8714, 1, 2022/06/17, 11:18:29.732, 9406716979570, datacollector.dll, [coverlet]Unable to instrument module: /Users/../src/bin/Debug/net6.0/redacted.dll, pdb without local source files, [/Users/../src/../OneOf.SourceGenerator/OneOf.SourceGenerator.OneOfGenerator/GenerateOneOfAttribute.cs]

Due to this only 2 of our 61 projects are able to be covered. We also use internal source generators. Relying on file suffix isn’t really an option when you start using 3rd parties.

[daveMueller]

I start working on this. But it could take some time 🙏.