Coverage report of built app doesn't map to source files

Link to bug demonstration repository

Repro repo

I think I followed the docs correctly, but it’s certainly possible I missed a step 🤣

Expected Behavior

According to this, running a coverage report for a built app should map back to original source files.

Steps taken in above repo:

1. Create empty app using create-react-app
2. Build app, which generates sourcemaps
3. Following these docs, instrument build output using nyc instrument ./static ./static-i
4. Swap static directories so instrumented code is served
5. Run automated tests, aggregating window.__coverage__
6. Following these docs, dump coverage data to .nyc_output/out.json
7. Run nyc report --reporter=html

Observed Behavior

The report shows coverage for built files, not source files.

Troubleshooting steps

• still occurring when I put cache: false in my nyc config

Environment Information

System:
    OS: macOS 10.14.5
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Memory: 242.63 MB / 16.00 GB
Binaries:
    Node: 10.13.0 - /usr/local/bin/node
    Yarn: 1.7.0 - /usr/local/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm

[evanbb]

@bcoe sure thing: https://github.com/evanbb/nyc-instrument-and-report-bug

I stepped into it a bit, and it looked like the reporting logic is trying to reload the sourcemaps from cache here:

https://github.com/istanbuljs/nyc/blob/master/index.js#L567

However, the fileReport doesn’t even have a contentHash property on it, so the check below fails:

https://github.com/istanbuljs/nyc/blob/master/lib/source-maps.js#L43

Maybe something else needs to be done during instrumentation that I am missing. I stepped through the instrumentation command a bit as well, and although the sourcemaps were correctly found on disk during instrumentation, there was nothing written to the node_modules/.cache/ directory.

I hacked it and manually wrote the corresponding files where they would have been in the cache directory, which caused the report to generate (almost) correctly: the report excluded node_modules and the root index.html file reported correct numbers for original source files, but viewing the coverage page for a single source file displayed the error below:

Unable to lookup source: ... (ENOENT: no such file or directory, open '...')
  at Context.defaultSourceLookup [as sourceFinder] (/node_modules/istanbul-lib-report/lib/context.js:15:15)
    at Context.getSource (/node_modules/istanbul-lib-report/lib/context.js:78:17)
    at Object.annotateSourceCode (/node_modules/istanbul-reports/lib/html/annotator.js:217:40)
    at HtmlReport.onDetail (/node_modules/istanbul-reports/lib/html/index.js:265:27)
    at Visitor.(anonymous function) [as onDetail] (/node_modules/istanbul-lib-report/lib/tree.js:34:30)
    at ReportNode.Node.visit (/node_modules/istanbul-lib-report/lib/tree.js:114:17)
    at getChildren.forEach.child (/node_modules/istanbul-lib-report/lib/tree.js:118:15)
    at Array.forEach ()
    at ReportNode.Node.visit (/node_modules/istanbul-lib-report/lib/tree.js:117:24)
    at Tree.visit (/node_modules/istanbul-lib-report/lib/tree.js:150:20)

So something is definitely still missing…

[ngocvantran]

I’m having the same problem.

When using nyc instrument, the window.__coverage__ is generated for the compiled files instead of source files. nyc report then shows empty report because the window.__coverage__ values does not have contentHash, only hash.

I’ve also tried adding the instrumentation during the build step using babel-plugin-istanbul. Using this approach, window.__coverage__ is generated for the correct source files. However, nyc report again shows empty report because even in this case, window.__coverage__ does not have contentHash value.

So I think we have two separate issues

• How to make nyc instrument process source maps?
• How to make nyc report accepts coverage data stored in window.__coverage__?

Edit: The issue with nyc report can be bypassed by setting --exclude-after-remap to false so that it will ignore the problem with source maps.