Build watch takes 30x as long as cold build, high transformTime

TypeScript Version: 3.7.3 and 3.4.5

Search Terms: transformTime, Watch, Slow compilation

Code

When building our source code, we can do a cold build in about 25 seconds, but a watch build takes 15 minutes to start on Typescript 3.4.5. On Typescript 3.7.3, the watch time doubles to 30 minutes (60x as slow).

Running extended diagnostics shows that the vast majority of the time (96%) is spent on “transformTime”. Is there any way to speed this up, as I do not consider our project too large on the grander scale (600k LOC).

One thing that was causing slow builds that I already fixed and ruled out was the inlined type definitions bug mentioned in #34119, which cut my build time by a third. But even after this, I still get a very slow transformTime.

Files:                  3043
Lines:                637432
Nodes:               2372135
Identifiers:          850259
Symbols:             1167934
Types:                462989
Memory used:        2036527K
I/O Read time:         1.03s
Parse time:            2.81s
Program time:          6.12s
Bind time:             3.05s
Check time:           11.54s
transformTime time:  473.05s
commentTime time:      0.14s
I/O Write time:        0.43s
printTime time:      474.27s
Emit time:           474.27s
Total time:          494.99s

Expected behavior:

Watch build is slightly slower than cold build

Actual behavior:

Watch build is significantly slower

Playground Link:

Related Issues:

[lingz]

I’ve managed to find out the culprit, and reduced the transform time from 473 seconds to 6 seconds.

It turns out it was a recursive type that was used only a few places in our code:

/**
 * The JSON version, without functions, methods, class
 */
export type LitNative = string | number | boolean | undefined | null | void;
export type WithoutFunctions<T> = T extends any
  ? Omit<T, FunctionKeys<Required<T>>>
  : T

export type Serialized<T> = T extends any[]
  ? InnerSerialized<T[number]>[]
  : (T extends LitNative ? T : InnerSerialized<WithoutFunctions<T>>);

type InnerSerialized<T> = {
  [k in keyof T]: T[k] extends any[]
    ? Serialized<T[k][number]>[]
    : Serialized<T[k]>
};

This type Serialized is recursive, and takes an object and gives a version of it without any functions (as if it went through JSON.serialize). It was only used a few places in the code but it was exploding the transformTime.

The way I found it, was after seeing that trySymbolTable was taking up most of the time, I ran the tsc compiler with the chrome inspector, and set a breakpoint on the function. I saw all the function calls were looking up Serialized over and over again. I would speculate that this could be mitigated by some caching in the tsc architecture?

The command I ran (in case it is helpful to others) is:

node --inspect-brk $(which tsc) -p ./tsconfig.json --declaration --emitDeclarationOnly --extendedDiagnostics --declarationDir ~/Temp/declarations/out > ~/Temp/declarations/diagnostics.txt

[kumar303]

TL;DR it would be nice to have better diagnostics tools to tell us which exact types are the slowest (and what file / module they come from)

After my own debugging adventure, I tracked down the slowness to a third party library type. Because of that, the above techniques didn’t help me discover it. Here was my process, in case it helps someone else.

For context, I was converting an 80k LOC monorepo from Flow to TypeScript. When editing any file during tsc --watch, it was taking 57 seconds to re-compile (every time).

• I checked my source for large declaration files as mentioned in https://github.com/microsoft/TypeScript/issues/35729#issuecomment-582198756 (thanks @ConneXNL and @lingz) – I didn’t find any.
• I tried moving my tsconfig.json to subdirectories in my source to narrow down the files – no speedups
• I tried a composite layout – no speedups
• I did a find/replace of type to interface – no noticeable speedup
• I wrote a jscodeshift transform that adds // @ts-nocheck to all files in a directory. I ran this across several directories to try and eliminate types – no speedups anywhere.

code mod to add @ts-nocheck

// I named this ./codemods/transforms/add-ts-nocheck.ts
//
// I ran it from ./codemods like:
// jscodeshift -t ./transforms/add-ts-nocheck.ts ../my-actual-project --extensions=tsx --parser=tsx

import { Transform } from "jscodeshift";

/*
 * Add a @ts-nocheck comment to the top of every file.
 */
const transform: Transform = function (fileInfo, api) {
  const filePath = fileInfo.path ?? "";

  // I was eliminating types by directory like this:
  // if (!filePath.startsWith("../my-actual-project/packages/some-package")) {
  //   return null;
  // }
  if (filePath.includes("node_modules")) {
    return null;
  }

  const root = api.jscodeshift(fileInfo.source);
  root.get().node.program.body.unshift("// @ts-nocheck");
  return root.toSource();
};

transform.parser = "ts";
export default transform;

• After adding // @ts-nocheck to literally every file in my source, I began to wonder if it even had an effect on compilation at all but I also wondered if it could be a third party typing problem.
• My first guess was reakit since we were on an ancient version written in TypeScript (I dunno, just a sixth sense). I added this to @types/reakit/index.d.ts to override all its types:

@types/reakit/index.d.ts

declare module "reakit" {
  var x: any;
  export = x;
}

This brought my watch mode re-compiles down from 57 seconds to 3 seconds ✨ 🌈 🥇

I tried narrowing down the exact type in reakit like this:

@types/reakit/index.d.ts

import {
  Provider,
  // Tooltip,
} from "reakit";

declare module "reakit" {
  // Replace just this one export:
  var Tooltip: any;

  var allExports = {
    Provider,
    Tooltip,
    // Re-add all original exports ...
  };
  export = allExports;
  export default allExports;
}

…but I couldn’t find a single export that was slow. They were all slow. I guess there is a shared internal type somewhere causing the problem.

If I hadn’t gotten lucky and guessed the bad third party lib, I probably would have written a script to override all third party types then re-enable them one by one. I looked for an easier way to do that in TS but couldn’t find one.

I was really in the dark here so better diagnostic tools would have been super helpful.