Build is significantly slower with a large node_modules folder

Steps to reproduce

In an empty folder, run dotnet new. Change the generated project.json file to:

{
  "version": "1.0.0-*",
  "buildOptions": {  "emitEntryPoint": true  },
  "dependencies": {  },
  "frameworks": {  "net461": {}  }
}

Run dotnet restore; then run dotnet build and observe that the build banner (first two lines below) are displayed instantly, and the build finishes in less than 2 seconds:

Project Test (.NETFramework,Version=v4.6.1) will be compiled because expected outputs are missing
Compiling Test for .NETFramework,Version=v4.6.1

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:01.8254959

In the folder containing project.json, create an NPM package.json file with the following content:

{
  "dependencies": {
    "babel-core": "^6.8.0",
    "babel-loader": "^6.2.4",
    "babel-plugin-transform-runtime": "^6.8.0",
    "babel-preset-es2015": "^6.6.0",
    "babel-preset-react": "^6.5.0",
    "babel-preset-stage-0": "^6.5.0",
    "bootstrap": "^4.0.0-alpha.2",
    "classnames": "^2.2.5",
    "css-loader": "^0.23.1",
    "es6-promise": "^3.1.2",
    "exports-loader": "^0.6.3",
    "extract-text-webpack-plugin": "^1.0.1",
    "file-loader": "^0.8.5",
    "font-awesome": "^4.6.1",
    "imports-loader": "^0.6.5",
    "node-sass": "^3.7.0",
    "query-string": "^4.1.0",
    "react": "^15.0.2",
    "react-dom": "^15.0.2",
    "react-router": "^2.4.0",
    "react-select": "1.0.0-beta13",
    "react-spinkit": "^1.1.7",
    "sass-loader": "^3.2.0",
    "source-map-loader": "^0.1.5",
    "style-loader": "^0.13.1",
    "ts-loader": "^0.8.2",
    "typescript": "^1.8.10",
    "typings": "^0.8.1",
    "url-loader": "^0.5.7",
    "webpack": "^2.1.0-beta.6"
  }
}

Run npm install. With NPM 3.8.8 and Node 6.0, this creates 2095 sub-folders containing more than 20000 files under the node_modules folder. This is an example web application development scenario, but the issue reproduces whenever a large number of files/folders are present in the project folder.

Run dotnet build and note that there is a significant delay before the any output is displayed (on my machine, around 5-6 seconds with CPU usage above 20%). The build takes a lot longer (3x):

Project Test (.NETFramework,Version=v4.6.1) will be compiled because expected outputs are missing
Compiling Test for .NETFramework,Version=v4.6.1

Compilation succeeded.
    0 Warning(s)
    0 Error(s)

Time elapsed 00:00:07.1625644

After the build output is displayed, there is another large pause (>5 seconds) before the command prompt returns.

Update project.json to exclude the node_modules folder from the compilation:

  "buildOptions": {
    "emitEntryPoint": true,
    "compile": {
        "exclude": ["node_modules"]
    }
  },

Running dotnet build hits dotnet/cli#2841 and the build fails silently. Update project.json again to include .cs files in the compilation:

  "buildOptions": {
    "emitEntryPoint": true,
    "compile": {
        "include": ["**/*.cs"],
        "exclude": ["node_modules"]
    }
  },

There is still a large pause before and after the command output, but the build finished somewhat faster (2x slowdown). A unexpected warning is now displayed about the obj\assembly.cs file:

Project Test (.NETFramework,Version=v4.6.1) will be compiled because Input items added from last build
Compiling Test for .NETFramework,Version=v4.6.1
D:\projects\Test\warning CS2002: Source file 'obj\Debug\net461\dotnet-compile.assemblyinfo.cs' specified multiple times

Compilation succeeded.
    1 Warning(s)
    0 Error(s)

Time elapsed 00:00:04.6567803

Update project.json to exclude the obj folder:

  "buildOptions": {
    "emitEntryPoint": true,
    "compile": {
        "include": ["**/*.cs"],
        "exclude": ["node_modules", "obj"]
    }
  },

The pauses before/after are the same, the build completes in 4.5 seconds, but the warning is gone.

Expected behavior

The build slowdown caused by a large node_modules folder can be avoided by adding node_modules to compile/exclude.

Actual behavior

Build remains significantly slower when node_modules is added to compile/exclude.

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-rc2-002611)

Product Information:
 Version:     1.0.0-rc2-002611
 Commit Sha:  bf8f0edd89

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64

[GunterGlad]

I have just found the same issue. Our node_modules is pretty big. It is supposed to be excluded: “buildOptions”: { “compile”: { “exclude”: [ “node_modules” ] } }, My compile time was 2m30. I deleted the node_modules folder and my compile time wend down to 30s.

[rhyek]

Related: dotnet/cli#5656