Sucrase can't transpile d3-array

Over here at @toucantoco we’re using Sucrase and we’re quite happy with it, thanks for maintaining it! However after a deps update we can’t run the tests anymore because it looks like Sucrase can’t properly transpile this file

As you can see in this Sucrase playground by checking and unchecking this typescript option, Sucrase fails when TS is on

However this is valid TS, and as you can see in this TS playground, this properly transpiles to JS, and if noImplicitAny is set to false there is no error

I don’t know why this happens when running our tests since @sucrase/jest-plugin precises that a .js will not be transpiles with the typescript option

Anyway just wanted to give you a heads up on this, I have no idea how to fix it yet but I can try if you guys are too busy

[alangpierce]

Thanks for reporting, I’m getting back to Sucrase maintenance now and this is definitely a tricky one! Working on it now and I’m not totally sure how to fix, but I figure writing down some thoughts may help.

Babel ran into the same bug and fixed it here: https://github.com/babel/babel/pull/13581/files . Unfortunately, that fix is a rare situation where the Babel parser inspects its partially-created AST and uses it to make parsing decisions. Sucrase intentionally avoids creating an AST, so there isn’t an obvious equivalent of the fix for Sucrase.

In general, arrow functions are among the hardest things to parse in JS, and TS makes them even more complicated. If you’re reading the code token-by-token and see (a = 1, b = 2), there’s no way to know if you’re dealing with two variable assignments separated by the comma operator, or if this is the start of an arrow function with two parameters with default values. It’s only when seeing the next token after the ) that you can distinguish those two situations; for example, if you see a => token, then this must actually be an arrow function. Babel’s strategy is take the incorrectly-parsed comma operator AST and rewrite it so that it instead represents arrow parameters. Sucrase doesn’t have an AST but still needs to correctly indicate IdentifierRole for the params, so it does a restore-from-snapshot to before the ( and re-parses the expression as an arrow function. In both cases, the relevant function is parseParenAndDistinguishExpression. TypeScript syntax needs extra care because : after a parenthesized list might indicate the return type of the arrow function.

In the example failing code, Sucrase (and Babel before the bug fix) is seeing (PARAMETERS) : a => 0, which is a valid arrow expression where a is the return type. Babel’s fix is to inspect the AST for PARAMETERS, see that a += 0 isn’t actually a valid parameter declaration, and bail out of the arrow parsing case. Sucrase might need to detect that distinction up-front or have a fallback when the arrow case later doesn’t parse correctly. I’ll also take a look at how the TypeScript parser handles it; it might take an approach that’s easier to incorporate into Sucrase.

[ninofiliu]

Allowing the Sucrase Jest plugin not to use the Flow transform on .js files would allow Sucrase Jest plugin users to bypass the issue

I have the bandwith to suggest a PR allowing Sucrase Jest plugin users to configure it as such

{
  "transform": {
    // uses default transforms
    "\\.js$": ["@sucrase/jest-plugin"],
    // uses custom transforms
    "\\.jsx$": ["@sucrase/jest-plugin", { "transforms": ["jsx", "imports", "jest"] }]
  }
}