Proposal: Use Node's native coverage when available or specified

Expected Behavior

1. nyc node test.js should work when test.js imports ES6 modules

2. Likewise nyc mocha and other similarly invoked test frameworks should work when the tests import ES6 modules.

3. Be able to generate coverage using tsc transpiled Typescript (or any other transpiler for that matter) rather than requiring the use of ts-node and --require ts-node/register.

   • Simpler
   • Does not force people with already established use of tsc to figure out how to also integrate ts-node into their builds just to get test coverage reports.
   • ts-node does not support ES6 modules.

4. Tools that integrate with nyc/Istanbul such as Webstorm inherit all of the above automatically.

nyc I believe was supposed to make Istanbul use simple and “just work”. In at least the above cases, and possible many others, @bcoe’s c8’s use of Node’s built-in coverage reporting fulfills that promise far better.

Observed Behavior

Empty coverage reports (a few lines all zeros) or errors.

Troubleshooting steps

I’ll not try to remember and record the day and a half of trial and error after error here because I’ve learned that this is a known issue, though I’m not sure if it’s a ts-node problem or an nyc limitation (because it can’t hook into into imports like it does require?)

I did confirm nyc is able to instrument the files using nyc instrument.

Then by chance on the Istanbul homepage I saw:

Like nyc, c8 just magically works:

npm i c8 -g
c8 node foo.js

I installed c8 and boom, it worked instantly, and yes, magically. Whether running a homemade test.js, Mocha based-tests, or even non-test code. No special nyc configuration for Typescript (it automatically picked up the source-maps), no need to install ts-node and inject it into the mocha command line (--require ts-node/register).

But only on the command line. I use Webstorm, which has specific integrations with nyc and so does not pick up the coverage report from c8 even though it reports using the same Istanbul’s reporters Webstorm reads.

Proposal

Merge c8 into nyc, or have nyc wrap c8 and it’s original code, so that all tools that work with nyc will transparently benefit from the optional improved compatibility of Node’s native coverage instrumentation.

I can wait and hope the Webstorm folks integrate with C8. But in my above hunt for a solution I’ve come across many other people in other situations whose woes I suspect would disappear if Node’s native functionality were leveraged, either optionally or automatically.

Environment Information

  System:
    OS: macOS 10.15.5
    CPU: (4) x64 Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz
    Memory: 81.98 MB / 8.00 GB
  Binaries:
    Node: 14.8.0 - /usr/local/bin/node
    npm: 6.14.8 - /usr/local/bin/npm
  npmPackages:
    eslint-plugin-babel: ^5.3.1 => 5.3.1 
    source-map-support: ^0.5.19 => 0.5.19 
    typescript: ^4.0.2 => 4.0.2 

[andersk]

@istanbuljs/esm-loader-hook no longer works in Node.js ≥ 16.12.0 due to an incompatible change in the experimental ESM loader hook API (istanbuljs/esm-loader-hook#6).

[coreyfarrell]

c8 is a great tool but it is a separate project. I’m going to close this issue as nyc will not be merging with c8, both tools have their place and in some niche cases c8 might not be best (likewise in some niche cases nyc is not best). I think the current situation is best where package maintainers choose which tool is best for their usage.