Extend Source Maps to support post-hoc debugging

Suggestion

🔍 Search Terms

Source maps, post hoc debugging

✅ Viability Checklist

My suggestion meets these guidelines:

• This wouldn’t be a breaking change in existing TypeScript/JavaScript code
• This wouldn’t change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn’t a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• This feature would agree with the rest of TypeScript’s Design Goals.

⭐ Suggestion

I would like to propose that TypeScript create an experimental source map extension (version 4). The purpose of this would be to improve the ability to apply Source Map data to stack traces and to have the Source Map contain everything needed to go from a minified stack trace to an unminified stack trace without using function name guessing.

📃 Motivating Example

Suppose I receive the following information from my application:

TypeError: Cannot read properties of undefined (reading 'x')
    at re.draw (bundle.js:1:16372)
    at re.drawLayer (bundle.js:1:14170)
    at be.draw (bundle.js:1:74592)
    at Te.render (bundle.js:1:114230)
    at Te.reflowCanvas (bundle.js:1:113897)
    at HTMLDocument.anonymous (bundle.js:1:135849)

The source map will resolve the first stack frame to draw(src, sprite, x, y, frame), and will correctly point out that the failure here is that the undefined value is actually frame. However, that is not the name of the function. The function’s name is draw, which is a member of the Render class.

If I simply apply the value of the names array to the function, decoding the stack trace is not particularly useful. It would look something like this:

TypeError: Cannot read properties of undefined (reading 'x')
    at frame (file1.ts:302:7)
    at draw (file1.ts:144:5)
    at Render (file2.ts:95:5)
    at drawArray (file3.ts:178:5)
    at render (file3.ts:155:5)
    at game (file4.ts:39:5)

Using the source map to navigate the source code by hand, I can reconstruct the original call stack:

TypeError: Cannot read properties of undefined (reading 'x')
    at Render#draw (file1.ts:302:7)
    at Render#drawLayer (file1.ts:144:5)
    at GameObject#draw (file2.ts:95:5)
    at Game#render (file3.ts:178:5)
    at Game#reflowCanvas (file3.ts:155:5)
    at [anonymous function passed to addEventListener] (file4.ts:39:5)

But doing this required that I dump the mappings and the source files from the source map and manually inspect the source files. In general, this requires that I use a library to parse the mappings field from Source Maps (because as of Source Maps v3, this field is stored in a stateful way).

💻 Use Cases

I want to use this to improve in-production debugging experiences, specifically, to improve stack traces, particularly those that exist after minification.

Presently, to work around this, we need to do one of two things:

• Manually reconstruct the stack by inspecting the source contents (as mentioned above)
• Or, by writing code that “guesses” the function name by walking backwards from the location the Mapping produces. (This is the mechanism that stacktrace-gps uses, although it often doesn’t work for TypeScript sources because it doesn’t recognize type annotations).

Proposed changes to the Source Map spec

Substantively: Only the mappings field would be altered, and would be altered by adding the 6th field. The complete section is included below:

The “mappings” data is broken down as follows:

• each group representing a line in the generated file is separated by a ”;”
• each segment is separated by a “,”
• each segment is made up of 1,4 or 5, 5, or 6 variable length fields.

The fields in each segment are:

1. The zero-based starting column of the line in the generated code that the segment represents. If this is the first field of the first segment, or the first segment following a new generated line (“;”), then this field holds the whole base 64 VLQ. Otherwise, this field contains a base 64 VLQ that is relative to the previous occurrence of this field. Note that this is different than the fields below because the previous value is reset after every generated line.
2. If present, an zero-based index into the “sources” list. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented.
3. If present, the zero-based starting line in the original source represented. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. Always present if there is a source field.
4. If present, the zero-based starting column of the line in the source represented. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. Always present if there is a source field.
5. If present, the zero-based index into the “names” list associated with this segment. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented. 6. If present, the zero-based index into the “names” list associated with the call stack of this segment. This field is a base 64 VLQ relative to the previous occurrence of this field, unless this is the first occurrence of this field, in which case the whole value is represented.

In addition, the version field of the spec should be bumped to 4.

Suggested names

• If functions are named directly, they should be preserved (function foo() { ... }, const foo = () => { ... }, { foo: function() { … }`
• If the function name cannot be deduced from an assignment, it should either be [anonymous function] or [anonymous arrow function]
• If the function appears to be passed as a callback, and the expression being called can be identified, its name should be used: [anonymous function passed to addEventListener]
• If the area is not contained within a function, it should be the name Global code

[robpalme]

Hello friends, this is a valuable topic. I’m pleased to see interest in progressing the sourcemap spec.

We’re currently using the pasta-sourcemaps extension (originally created by @ldarbi) to deal with this. Pasta stands for “Pretty (and) Accurate Stack Trace Analysis”.

The extension solves the exact use-case outlined above. It permits accurate decoding of function names without guessing and without the need to consult the original source files. The sourcemap tells you everything.

pasta-sourcemaps works by adding an additional field to the sourcemap called "x_com_bloomberg_sourcesFunctionMappings" which contains a series of VLQ-encoded mappings that identify named function spans in the original source. So you first use a pre-existing decoding function (that reads "mappings") to identify a source position (file, line, column), and then use that position to lookup the original function name in the dedicated list of function spans. Here’s the spec.

pasta-sourcemaps has been used in production as part of the Bloomberg Terminal’s crash stack telemetry for over two years, handling millions of stacks. It supports .js/.jsx/.ts/.tsx source files, is mature, and has a nifty logo. We aim to keep it up to date with the latest TypeScript version - though it is temporarily lagging on TS 4.3.

We had early discussions with @bcoe about getting pasta support into Node but never got around to acting on it. Since then, Node gained a best-effort (guessing) implementation, but would still benefit from something like this to increase reliability.

Please take a look at the approach for inspiration. It would be interesting to compare extending the "mappings" vs adding an extra field. We’ve not been in control of the tools (like TypeScript) that generate "mappings" and therefore it seemed easiest to add an extra field that can be guaranteed to be a complete record of all the necessary function ranges. Whereas the sourcemap spec itself says nothing about completeness of "mappings" - it’s up to the specific encoder (e.g. in TypeScript, or in Terser) to decide when to emit them, so different tools make different decisions on the fidelity of the points.

So thank you @robpaveza for raising this issue! I’d love to see this problem more widely solved. Beyond stack trace decoding, this information could also be used in DevTools, e.g. the VSCode debugger’s call stack could use it to show the original function name when debugging minified code.

[bcoe]

I would be interested in being looped in to this work, if an effort is made to dust off the Source Map spec. I would love to make sure Istanbul and Node.js both support extensions.