question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Extend Source Maps to support post-hoc debugging

See original GitHub issue

Suggestion

🔍 Search Terms

Source maps, post hoc debugging

✅ Viability Checklist

My suggestion meets these guidelines:

  • This wouldn’t be a breaking change in existing TypeScript/JavaScript code
  • This wouldn’t change the runtime behavior of existing JavaScript code
  • This could be implemented without emitting different JS based on the types of the expressions
  • This isn’t a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
  • This feature would agree with the rest of TypeScript’s Design Goals.

⭐ Suggestion

I would like to propose that TypeScript create an experimental source map extension (version 4). The purpose of this would be to improve the ability to apply Source Map data to stack traces and to have the Source Map contain everything needed to go from a minified stack trace to an unminified stack trace without using function name guessing.

📃 Motivating Example

Suppose I receive the following information from my application:

TypeError: Cannot read properties of undefined (reading 'x')
    at re.draw (bundle.js:1:16372)
    at re.drawLayer (bundle.js:1:14170)
    at be.draw (bundle.js:1:74592)
    at Te.render (bundle.js:1:114230)
    at Te.reflowCanvas (bundle.js:1:113897)
    at HTMLDocument.anonymous (bundle.js:1:135849)

The source map will resolve the first stack frame to draw(src, sprite, x, y, frame), and will correctly point out that the failure here is that the undefined value is actually frame. However, that is not the name of the function. The function’s name is draw, which is a member of the Render class.

If I simply apply the value of the names array to the function, decoding the stack trace is not particularly useful. It would look something like this:

TypeError: Cannot read properties of undefined (reading 'x')
    at frame (file1.ts:302:7)
    at draw (file1.ts:144:5)
    at Render (file2.ts:95:5)
    at drawArray (file3.ts:178:5)
    at render (file3.ts:155:5)
    at game (file4.ts:39:5)

Using the source map to navigate the source code by hand, I can reconstruct the original call stack:

TypeError: Cannot read properties of undefined (reading 'x')
    at Render#draw (file1.ts:302:7)
    at Render#drawLayer (file1.ts:144:5)
    at GameObject#draw (file2.ts:95:5)
    at Game#render (file3.ts:178:5)
    at Game#reflowCanvas (file3.ts:155:5)
    at [anonymous function passed to addEventListener] (file4.ts:39:5)

But doing this required that I dump the mappings and the source files from the source map and manually inspect the source files. In general, this requires that I use a library to parse the mappings field from Source Maps (because as of Source Maps v3, this field is stored in a stateful way).

💻 Use Cases

I want to use this to improve in-production debugging experiences, specifically, to improve stack traces, particularly those that exist after minification.

Presently, to work around this, we need to do one of two things:

  • Manually reconstruct the stack by inspecting the source contents (as mentioned above)
  • Or, by writing code that “guesses” the function name by walking backwards from the location the Mapping produces. (This is the mechanism that stacktrace-gps uses, although it often doesn’t work for TypeScript sources because it doesn’t recognize type annotations).

Proposed changes to the Source Map spec

Substantively: Only the mappings field would be altered, and would be altered by adding the 6th field. The complete section is included below:

The “mappings” data is broken down as follows:

  • each group representing a line in the generated file is separated by a ”;”
  • each segment is separated by a “,”
  • each segment is made up of 1,4 or 5, 5, or 6 variable length fields.

The fields in each segment are:

  1. The zero-based starting column of the line in the generated code that the segment represents. If this is the first field of the first segment, or the first segment following a new generated line (“;”), then this field holds the whole base 64 VLQ. Otherwise, this field contains a base 64 VLQ that is relative to the previous occurrence of this field. Note that this is different than the fields below because the previous value is reset after every generated line.
  2. If present, an zero-based index into the “sources” list. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented.
  3. If present, the zero-based starting line in the original source represented. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. Always present if there is a source field.
  4. If present, the zero-based starting column of the line in the source represented. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. Always present if there is a source field.
  5. If present, the zero-based index into the “names” list associated with this segment. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. 6. If present, the zero-based index into the “names” list associated with the call stack of this segment. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented.

In addition, the version field of the spec should be bumped to 4.

Suggested names

  • If functions are named directly, they should be preserved (function foo() { ... }, const foo = () => { ... }, { foo: function() { … }`
  • If the function name cannot be deduced from an assignment, it should either be [anonymous function] or [anonymous arrow function]
  • If the function appears to be passed as a callback, and the expression being called can be identified, its name should be used: [anonymous function passed to addEventListener]
  • If the area is not contained within a function, it should be the name Global code

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:11 (3 by maintainers)

github_iconTop GitHub Comments

8reactions
robpalmecommented, Nov 6, 2021

Hello friends, this is a valuable topic. I’m pleased to see interest in progressing the sourcemap spec.

We’re currently using the pasta-sourcemaps extension (originally created by @ldarbi) to deal with this. Pasta stands for “Pretty (and) Accurate Stack Trace Analysis”.

The extension solves the exact use-case outlined above. It permits accurate decoding of function names without guessing and without the need to consult the original source files. The sourcemap tells you everything.

pasta-sourcemaps works by adding an additional field to the sourcemap called "x_com_bloomberg_sourcesFunctionMappings" which contains a series of VLQ-encoded mappings that identify named function spans in the original source. So you first use a pre-existing decoding function (that reads "mappings") to identify a source position (file, line, column), and then use that position to lookup the original function name in the dedicated list of function spans. Here’s the spec.

pasta-sourcemaps has been used in production as part of the Bloomberg Terminal’s crash stack telemetry for over two years, handling millions of stacks. It supports .js/.jsx/.ts/.tsx source files, is mature, and has a nifty logo. We aim to keep it up to date with the latest TypeScript version - though it is temporarily lagging on TS 4.3.

We had early discussions with @bcoe about getting pasta support into Node but never got around to acting on it. Since then, Node gained a best-effort (guessing) implementation, but would still benefit from something like this to increase reliability.

Please take a look at the approach for inspiration. It would be interesting to compare extending the "mappings" vs adding an extra field. We’ve not been in control of the tools (like TypeScript) that generate "mappings" and therefore it seemed easiest to add an extra field that can be guaranteed to be a complete record of all the necessary function ranges. Whereas the sourcemap spec itself says nothing about completeness of "mappings" - it’s up to the specific encoder (e.g. in TypeScript, or in Terser) to decide when to emit them, so different tools make different decisions on the fidelity of the points.

So thank you @robpaveza for raising this issue! I’d love to see this problem more widely solved. Beyond stack trace decoding, this information could also be used in DevTools, e.g. the VSCode debugger’s call stack could use it to show the original function name when debugging minified code.

2reactions
bcoecommented, Nov 8, 2021

I would be interested in being looped in to this work, if an effort is made to dust off the Source Map spec. I would love to make sure Istanbul and Node.js both support extensions.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Improving Development with Source Maps for Debugging
These source maps enable DevTools functionality like debugging with breakpoints and debugger; in the Sources tab. You can configure whether ...
Read more >
Map the processed code to your original source code, for ...
Source maps map your compiled, minified code to your original source code files. In DevTools, you can then read and debug your familiar, ......
Read more >
Source maps in Node.js. Supporting the many flavors of…
Applying source maps to stack traces​​ ts:13:7 . This makes it much easier to debug exceptions in alternate flavors of JavaScript, such as ......
Read more >
US20020091702A1 - Dynamic object-driven database manipulation ...
The present invention provides a system and method for dynamic object-driven database manipulation and mapping system which relates in general to ...
Read more >
Map Preprocessed Code to Source Code - Chrome Developers
# Summary · Use Source Maps to map minified code to source code. You can then read and debug compiled code in its...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found