Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Slow design-time builds for large solution
See original GitHub issueIssue Description
Hi,
productivity in our company is heavily hindered by the slowness of design-time builds which translates into the slowness of solution load (and reload) times in the IDE (Visual Studio and Rider).
For example, we are working with solution containing about 270 C# and F# projects and use Central Package Management to manage version of our NuGet dependencies. The load time of such solution is about 2-3 minutes which is far too long to stay focused.
The fact that we use a mix of C# and F# projects is probably irrelevant because according to my internal testing the solution load time when all projects are C# projects is the same. Also, the usage of central package version management seems to be not really affecting how long it takes to run an individual design-time build. But the concerning consequence of using central package management is that any slightest change in the central file (Directory.Packages.Props) triggers a reload of all projects in the solution so the slowness of design-time builds is exacerbated by central package management.
We’ve prepared a skeleton version of our solution https://github.com/marcin-krystianc/TestSolutions/tree/master/LargeAppWithPrivatePackagesCentralisedNGBVRemoved which can be used for testing.
I don’t think that the problem is unique to our solution though, my suspicion (confirmed with some experiments) is that any solution of similar size has similar performance characteristics. So, given that the problem is relevant to all large solutions, we are wondering whether there are any plans to improve the performance of design-time builds for large solutions? Maybe a static graph feature can help? It turned out to be a huge win for NuGet restores (https://devblogs.microsoft.com/visualstudio/performance-improvements-in-nuget/#msbuild-static-graph-evaluation) so maybe it is possible to apply it for design-time builds as well?
Are you able to provide any hints on where to look for any improvements? Looking at text logs or binary logs (/bl) didn’t give any obvious answers so far.
Steps to Reproduce
- checkout https://github.com/marcin-krystianc/TestSolutions/tree/master/LargeAppWithPrivatePackagesCentralisedNGBVRemoved
- go to the
LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solutiondirectory - run
dotnet build - Load the solution in IDE or run:
dotnet msbuild /t:GetSuggestedWorkloads;_CheckForInvalidConfigurationAndPlatform;ResolveReferences;ResolveProjectReferences;ResolveAssemblyReferences;ResolveComReferences;ResolveNativeReferences;ResolveSdkReferences;ResolveFrameworkReferences;ResolvePackageDependenciesDesignTime;Compile;CoreCompile ^
/p:AndroidPreserveUserData=True ^
/p:AndroidUseManagedDesignTimeResourceGenerator=True ^
/p:BuildingByReSharper=True ^
/p:BuildingProject=False ^
/p:BuildProjectReferences=False ^
/p:ContinueOnError=ErrorAndContinue ^
/p:DesignTimeBuild=True ^
/p:DesignTimeSilentResolution=False ^
/p:JetBrainsDesignTimeBuild=True ^
/p:ProvideCommandLineArgs=True ^
/p:ResolveAssemblyReferencesSilent=False ^
/p:SkipCompilerExecution=True ^
/p:TargetFramework=net5.0 ^
/v:n ^
/m:1 ^
/bl ^
/flp:v=n;PerformanceSummary ^
/clp:Summary ^
/clp:PerformanceSummary > log.txt
Data
Sample logs from the command above. Please note, that this is not exactly the same as loading the solution in IDE, because IDE runs desgn-time build for each project individually wehereas we run it for entire solution all at once:
...
851 ms d:\workspace\TestSolutions\LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solution\Project53\Project53.fsproj 24 calls
0 ms GetSuggestedWorkloads 1 calls
7 ms _CheckForInvalidConfigurationAndPlatform 1 calls
600 ms ResolveReferences 1 calls
2 ms GetTargetFrameworks 6 calls
1 ms GetTargetPath 6 calls
0 ms ResolveProjectReferences 1 calls
0 ms ResolveAssemblyReferences 1 calls
0 ms ResolveComReferences 1 calls
0 ms ResolveNativeReferences 1 calls
0 ms ResolveSdkReferences 1 calls
0 ms ResolveFrameworkReferences 1 calls
161 ms ResolvePackageDependenciesDesignTime 1 calls
79 ms Compile 1 calls
0 ms CoreCompile 1 calls
1001 ms d:\workspace\TestSolutions\LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solution\Project12\Project12.fsproj 15 calls
0 ms GetSuggestedWorkloads 1 calls
7 ms _CheckForInvalidConfigurationAndPlatform 1 calls
791 ms ResolveReferences 1 calls
3 ms GetTargetFrameworks 1 calls
0 ms GetTargetFrameworksWithPlatformForSingleTargetFramework 1 calls
0 ms GetTargetPath 1 calls
0 ms ResolveProjectReferences 1 calls
0 ms ResolveAssemblyReferences 1 calls
0 ms ResolveComReferences 1 calls
0 ms ResolveNativeReferences 1 calls
0 ms ResolveSdkReferences 1 calls
0 ms ResolveFrameworkReferences 1 calls
127 ms ResolvePackageDependenciesDesignTime 1 calls
70 ms Compile 1 calls
0 ms CoreCompile 1 calls
1392 ms d:\workspace\TestSolutions\LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solution\Project276\Project276.csproj 12 calls
0 ms GetSuggestedWorkloads 1 calls
5 ms _CheckForInvalidConfigurationAndPlatform 1 calls
1113 ms ResolveReferences 1 calls
0 ms ResolveProjectReferences 1 calls
0 ms ResolveAssemblyReferences 1 calls
0 ms ResolveComReferences 1 calls
0 ms ResolveNativeReferences 1 calls
0 ms ResolveSdkReferences 1 calls
0 ms ResolveFrameworkReferences 1 calls
198 ms ResolvePackageDependenciesDesignTime 1 calls
76 ms Compile 1 calls
0 ms CoreCompile 1 calls
3875 ms d:\workspace\TestSolutions\LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solution\Project47\Project47.fsproj 26 calls
1 ms GetSuggestedWorkloads 1 calls
113 ms _CheckForInvalidConfigurationAndPlatform 1 calls
3426 ms ResolveReferences 1 calls
2 ms GetTargetFrameworks 7 calls
1 ms GetTargetPath 7 calls
0 ms ResolveProjectReferences 1 calls
0 ms ResolveAssemblyReferences 1 calls
0 ms ResolveComReferences 1 calls
0 ms ResolveNativeReferences 1 calls
0 ms ResolveSdkReferences 1 calls
0 ms ResolveFrameworkReferences 1 calls
224 ms ResolvePackageDependenciesDesignTime 1 calls
105 ms Compile 1 calls
0 ms CoreCompile 1 calls
125614 ms d:\workspace\TestSolutions\LargeAppWithPrivatePackagesCentralisedNGBVRemoved\solution\LargeAppWithPrivatePackagesCentralisedNGBVRemoved.sln 1 calls
125614 ms GetSuggestedWorkloads;_CheckForInvalidConfigurationAndPlatform;ResolveReferences;ResolveProjectReferences;ResolveAssemblyReferences;ResolveComReferences;ResolveNativeReferences;ResolveSdkReferences;ResolveFrameworkReferences;ResolvePackageDependenciesDesignTime;Compile;CoreCompile 1 calls
Target Performance Summary:
0 ms ValidateProjects 1 calls
0 ms ValidateToolsVersions 1 calls
2 ms GenerateMSBuildEditorConfigFile 155 calls
2 ms InitializeSourceControlInformation 276 calls
3 ms ResolvePackageDependenciesForBuild 276 calls
3 ms GetReferenceAssemblyPaths 276 calls
3 ms AddSourceRevisionToInformationalVersion 276 calls
3 ms GenerateAssemblyInfo 276 calls
3 ms GetTargetPath 212 calls
3 ms SetWin32ManifestProperties 276 calls
4 ms AfterCompile 276 calls
4 ms PrepareProjectReferences 276 calls
4 ms GetFrameworkPaths 276 calls
5 ms _SetTargetFrameworkMonikerAttribute 155 calls
5 ms BeforeResolveReferences 276 calls
6 ms _ReportUpgradeNetAnalyzersNuGetWarning 155 calls
6 ms AfterResolveReferences 276 calls
6 ms EnableIntermediateOutputPathMismatchWarning 276 calls
7 ms BeforeCompile 276 calls
7 ms _AddOutputPathToGlobalPropertiesToRemove 276 calls
8 ms _BeforeVBCSCoreCompile 155 calls
8 ms _CheckForUnsupportedHostingUsage 276 calls
9 ms _GetProjectJsonPath 276 calls
9 ms _DefaultMicrosoftNETPlatformLibrary 276 calls
9 ms CollectPackageReferences 276 calls
10 ms _CheckForUnsupportedAppHostUsage 70 calls
10 ms ExpandSDKReferences 276 calls
10 ms ResolveLockFileAnalyzers 276 calls
11 ms _CheckForObsoleteDotNetCliToolReferences 276 calls
12 ms ValidateSolutionConfiguration 1 calls
12 ms ValidateCommandLineProperties 276 calls
13 ms AddImplicitDefineConstants 276 calls
14 ms _CheckForUnsupportedCppNETCoreVersion 276 calls
14 ms _CheckForUnsupportedNETCoreVersion 276 calls
16 ms _CheckForLanguageAndFeatureCombinationSupport 276 calls
18 ms GenerateProgramFile 70 calls
18 ms GenerateMSBuildEditorConfigFileShouldRun 155 calls
19 ms CollectFSharpDesignTimeTools 121 calls
23 ms GetTargetPathWithTargetPlatformMoniker 212 calls
27 ms _ComputePackageReferencePublish 276 calls
28 ms _ComputeSkipAnalyzers 155 calls
35 ms _ComputeTargetFrameworkItems 144 calls
49 ms AddGlobalAnalyzerConfigForPackage_MicrosoftCodeAnalysisNetAnalyzers 155 calls
51 ms _GenerateCompileInputs 276 calls
55 ms GetAssemblyVersion 276 calls
58 ms _CollectTargetFrameworkForTelemetry 276 calls
58 ms CheckForImplicitPackageReferenceOverrides 276 calls
62 ms _SetEmbeddedWin32ManifestProperties 276 calls
68 ms GetTargetFrameworks 212 calls
72 ms GetAssemblyAttributes 276 calls
84 ms GenerateFSharpTextResources 121 calls
86 ms GetTargetFrameworksWithPlatformForSingleTargetFramework 212 calls
91 ms ResolveSDKReferences 277 calls
109 ms ApplyImplicitVersions 276 calls
111 ms _GetRestoreProjectStyle 276 calls
118 ms CheckForDuplicateItems 276 calls
121 ms UpdateAspNetToFrameworkReference 276 calls
124 ms IncludeTransitiveProjectReferences 276 calls
131 ms _GetFrameworkAssemblyReferences 121 calls
142 ms ValidateExecutableReferences 206 calls
143 ms GenerateMSBuildEditorConfigFileCore 155 calls
195 ms PrepareForBuild 276 calls
216 ms ResolveNativeReferences 1 calls
240 ms ResolveComReferences 1 calls
288 ms GenerateNETCompatibleDefineConstants 276 calls
293 ms CreateGeneratedAssemblyInfoInputsCacheFile 276 calls
337 ms CoreGenerateAssemblyInfo 276 calls
348 ms GenerateTargetFrameworkMonikerAttribute 276 calls
420 ms ResolveFrameworkReferences 277 calls
494 ms GetTargetFrameworksWithPlatformFromInnerBuilds 144 calls
662 ms RedirectTPReferenceToNewRedistributableLocation 121 calls
733 ms _GenerateCompileDependencyCache 121 calls
735 ms ProcessFrameworkReferences 276 calls
760 ms ResolveLockFileCopyLocalFiles 276 calls
939 ms ResolveTargetingPackAssets 276 calls
1044 ms RedirectFSharpCoreReferenceToNewRedistributableLocation 121 calls
1123 ms ResolveLockFileReferences 276 calls
1146 ms ResolveAssemblyReferencesDesignTime 276 calls
1527 ms AssignProjectConfiguration 276 calls
2019 ms _SplitProjectReferencesByFileExistence 276 calls
2084 ms _CheckForInvalidConfigurationAndPlatform 277 calls
4153 ms ResolvePackageAssets 276 calls
4199 ms CoreCompile 277 calls
5660 ms ResolveProjectReferences 277 calls
7393 ms FindReferenceAssembliesForReferences 276 calls
8683 ms GetSuggestedWorkloads 1 calls
9185 ms _HandlePackageFileConflicts 276 calls
11768 ms _GetProjectReferenceTargetFrameworkProperties 276 calls
14755 ms Compile 277 calls
18153 ms ResolveAssemblyReferences 277 calls
31281 ms RunResolvePackageDependencies 276 calls
40710 ms ResolvePackageDependenciesDesignTime 277 calls
60595 ms ResolveReferences 277 calls
Task Performance Summary:
3 ms Message 1 calls
12 ms GetFrameworkPath 276 calls
13 ms FSharpEmbedResXSource 121 calls
17 ms FSharpEmbedResourceText 121 calls
24 ms GetAssemblyVersion 276 calls
24 ms FindAppConfigFile 276 calls
29 ms AllowEmptyTelemetry 276 calls
31 ms CheckForDuplicateFrameworkReferences 276 calls
31 ms CombineXmlElements 212 calls
32 ms CheckForImplicitPackageReferenceOverrides 276 calls
33 ms CombineTargetFrameworkInfoProperties 212 calls
45 ms ApplyImplicitVersions 276 calls
48 ms ResolveFrameworkReferences 276 calls
62 ms GenerateMSBuildEditorConfig 155 calls
67 ms GetRestoreProjectStyleTask 276 calls
69 ms ResolveAppHosts 276 calls
72 ms CheckForDuplicateItems 828 calls
83 ms CheckIfPackageReferenceShouldBeFrameworkReference 552 calls
124 ms ValidateExecutableReferences 206 calls
138 ms Hash 397 calls
153 ms GetPackageDirectory 2208 calls
163 ms MakeDir 397 calls
342 ms JoinItems 276 calls
343 ms WriteLinesToFile 397 calls
534 ms ProcessFrameworkReferences 276 calls
858 ms ResolveTargetingPackAssets 276 calls
1331 ms GetReferenceNearestTargetFrameworkTask 304 calls
1341 ms AssignProjectConfiguration 276 calls
1350 ms Fsc 121 calls
3434 ms PreprocessPackageDependenciesDesignTime 276 calls
3854 ms ResolvePackageAssets 276 calls
7038 ms ResolvePackageFileConflicts 276 calls
16436 ms ResolveAssemblyReference 276 calls
30768 ms ResolvePackageDependencies 276 calls
139515 ms MSBuild 736 calls
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:02:06.04
Analysis
- Using higher parallelism (
/m) doesn’t really make it much faster as it seems that msbuild utilised more machine resources, but at the same time it also does much more work as caching inside msbuild is less effective.
Versions & Configurations
- dotnet 6.0.300
- VS 17.2.3
Regression?
I don’t think so.
Issue Analytics
- State:
- Created a year ago
- Reactions:3
- Comments:17 (15 by maintainers)
Top Results From Across the Web
The Scourge of Design-Time Builds | Panopticon Central
For example, a common place where slow design-time builds show up is switching between configurations. If you switch a large solution, say, ...
Read more >design-time-builds.md - dotnet/project-system
Design-time builds are special builds that are launched by the project ... For a slow design-time, look for the target performance summary at...
Read more >Large C# and VB solutions load significantly faster in 15.5 ...
The window will show a list of targets and design time build time for each project. Identify slow projects and inspect their targets,...
Read more >Speed up ReSharper (and Visual Studio)
ReSharper's incremental build can considerably reduce build time, especially for large solutions. Code Editing | Context Actions: here you can ...
Read more >Improve start up time - Visual Studio (Windows)
Open the Performance Manager dialog box by choosing Help > Visual Studio Performance Manager from the menu bar. If an extension is slowing...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Looking at this more, it looks like
ResolvePackageDependenciesdoes normally run, but it does less work unlessEmitLegacyAssetsFileItemsis true.However, it still has to read the assets file. It uses an in-memory cache for this so it won’t read the same assets file multiple times in the same build, but it looks like this may still be a perf hit in this scenario with a lot of projects in the solution.
Most of the logic for reading the assets file is now in
ResolvePackageAssets, which maintains its own optimized cache of the results and won’t read the assets file at all unless the cache is out of date. So we may want to consider reading thePackageDefinitionsandPackageDependenciesitems inResolvePackageAssets, so thatResolvePackageDependenciesdoesn’t need to be called at all unlessEmitLegacyAssetsFileItemsis true. That should improve incremental build perf and might also significantly improve the design time build perf for this large solution scenario.@dotnet/msbuild @marcpopMSFT
This should be fixed in 7.0.200 via #28405.
Thanks a bunch @marcin-krystianc for the detailed information which enabled us to investigate and address this!