Performance impact by searching excluded directories

Steps to reproduce

• Create a web project
• have a large folder with thousands of files in it (ie. node_modules)
• exclude the folder in the .csproj file for performance reasons

  <ItemGroup>
    <Compile Remove="node_modules\**" />
    <Content Remove="node_modules\**" />
    <EmbeddedResource Remove="node_modules\**" />
    <None Remove="node_modules\**" />
  </ItemGroup>

Expected behavior

• the folder should be ignored by the dotnet command
• project should load really fast in Visual Studio and JetBrains Raider

Actual behavior

• the folder is searched recursively
  • (issue was determined using the Sysinternals Process Monitor)
• project takes way too long to load
• creating/renaming/refactoring files and folders in Visual Studio takes way longer
• more RAM consumption

Environment data

dotnet --info output:

.NET Command Line Tools (2.0.0)

Product Information:
 Version:            2.0.0
 Commit SHA-1 hash:  cdcd1928c9

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.0.0\

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

[dasMulli]

The problem here is that the items are added beforehand so to exclude the folder being searched is to add it to the DefaultItemExcludes property like the web SDK already does for ASP.NET Core project.

To do it in non ASP.NET Core projects, you’d have to add a similar property redefinition to your csproj:

<PropertyGroup>
  <DefaultItemExcludes>node_modules/**;$(DefaultItemExcludes)</DefaultItemExcludes>
</PropertyGroup>

Note that there is a performance issue in glob expansion with patterns like **/node_modules/** at the moment so you’ll have to add the folders explicitly (best at the start of the DefaultItemExcludes as shown above)

[Piedone]

Got it, thank you for the quick reply and the thorough explanation. So if I understand correctly (and my short tests seem to confirm this) if you care about performance, and you want Visual Studio and MSBuild to basically not know about certain files then the thing you want to use, and you don’t need anything else, is DefaultItemExcludes, right? So no need for (or any benefit to) also use e.g. the following for git file? <EmbeddedResource Remove=".git" /> and <None Remove=".git" />. I’d use something like this:

  <PropertyGroup>
    <DefaultItemExcludes>$(DefaultItemExcludes);node_modules\**</DefaultItemExcludes>
  </PropertyGroup>

I wish it wasn’t necessary to have a PhD in MSBuild to use Node but working on my thesis now 😃.