Project roadmap and ways others can contribute

I’ve had a few requests from others about how they can help out. I’ve been keeping a checklist of possible future tasks, and figured it would be good to make that public to make it easier for others to contribute.

Feel free to stop by and chat in the gitter room, which I just created. Generally my thought is that this can be a big checklist of possible things to do, and as work starts on them, they can be “promoted” to real issues with individual discussions/assignees/etc. I’m sure there’s plenty of context to be shared about most of these issues, so feel free to chat if you want to take one of them on.

Currently, I’m focused on performance, and want to see how high I can get the N in “Sucrase is N times faster than Babel”. But there’s certainly lots of other stuff to do as well.

Stability

• Add additional example projects that can be automatically tested. Ideally they would be large codebases in TypeScript/Flow/JSX and have fast test suites so they can be run as part of PR builds.
• Find a way to integrate the Babylon test suite (and maybe other test suites like the TypeScript test suite and test262). I worry that there are some subtle edge cases around arrow function parsing that are missed right now.
• Set up istanbul and make sure the tests have good code coverage.

Bugs

• Complex assignments to mutable exports aren’t handled correctly, e.g. export let x; [x] = arr;. Babel and TypeScript are both incomplete here, and ideally we just mimic one or the other.
• TypeScript import pruning does not yet handle shadowed imported names.
• import statements should be hoisted to the top of the file, but they aren’t right now.
• The line numbers almost always match the source line numbers, but that doesn’t happen correctly for multiline class field initializers.
• There are subtle differences between Babel and TypeScript around the transformed result of import and export. Ideally we exactly copy either Babel or TypeScript, based on the config.
• Fix the disabled tests, which both crash the parser for TypeScript.
• Class field computed names with side-effects should be evaluated only once instead of on each constructor invocation.
• react-display-name has a case where it uses the filename to determine the display name for an anonymous export default React class. We could add a filename arg to the Sucrase API to support this, although it’s maybe obscure enough that the API change isn’t worth it.

Tooling

• Figure out automated releases. semantic-release might work, but it’s less clear since it’s a monorepo (each integration is its own package).
• Auto-publish the website on every release.
• Set up Greenkeeper or similar.
• See if Lerna would be useful. At the moment, the only packages other than the main sucrase package are small integrations, so Lerna may make less sense than for a larger project like Babel.
• Enable noImplicitThis in the TypeScript config. Implicit any for this has bit me a few times, but there are enough pre-existing errors that it’s hard to enable.
• Figure out a good way for Sucrase to compile itself for production builds. At the moment, there’s a self-build task, but we still use TypeScript for the real build.

Website (sucrase.io)

• It would be sweet to have a “file a bug” button that you can click to automatically submit a bug report with the current code. Even better if it can notice that Sucrase crashed and Babel succeeded, or Sucrase produced invalid JS, and explicitly prompt you to file a bug.
• Port the website to TypeScript or Flow and use Sucrase to build it.
• I’m sure there are plenty of visual polish improvements that could be made.
• Better handle Sucrase getting into an infinite loop, e.g. by adding a generous timeout (maybe 10 seconds) and restarting the web worker and giving a friendly error message.
• Improve the initial load experience. The Monaco editor is a fairly large download, and the web worker is also huge (1.1MB compressed), possibly due to babel-standalone being big. One approach is to optimize the bundle size, and another approach is to degrade better, e.g. showing a dumb editor if Monaco isn’t loaded and only running Sucrase if Babel isn’t loaded.
• Make the website more mobile-friendly.
• I recently disabled service workers because they were causing cache issues, but it would be nice to re-enable them.

Profiling/benchmarking

• Find a way to make performance numbers more stable, e.g. a dedicated server with nothing else running.
• Come up with a more realistic benchmark to test on. The current one is a bit boring/repetitive and may not be representative of real code.
• Explore other profiling techniques, like figuring out a way to measure the cost of function call overhead or other overhead, or looking at any JIT diagnostics provided by V8.
• It would be awesome to have a part of the website showing historical performance data, similar to https://arewefastyet.com/ . For example, it could run the Babel and Sucrase build on every example project every day and show how they compare.

Features

• ~Figure out if we need source maps, and implement them if so. The output line numbers are the same as the input line numbers, so theoretically they shouldn’t be necessary at least for things like stack traces.~ (Spun off as #223.)
• Support the TypeScript and Flow transforms with the imports transform disabled. (In other words, targeting ES modules).
• ~Add a transform for object rest/spread. I was hoping to avoid this, but it looks like Webpack uses a parser that doesn’t support it, so it may be necessary to implement for practical reasons. Also, MDN says it’s not supported at all in Edge or Safari. (Note that webpack-object-rest-spread-plugin makes this work in Webpack, and object will probably be very hard to implement in nested cases, so probably this transform should be called out of scope.)~ I’m calling this out of scope.

Configuration

• Allow Sucrase to read a tsconfig.json for its config (including which directories to compile and which to exclude, etc).
• Allow Sucrase to read a .babelrc for its config.
• Think more about how to simplify the configuration. One alternate approach is to specify the language being compiled (TS, TS+JSX, Flow, Flow+JSX, JSX) and the target (ESM, CJS) rather than a list of transforms. Much of this can be inferred from context, though, so ideally in most situations there can be zero configuration.
• Another idea I’ve had is “Sucrase Surgeon”, which goes into your node_modules directory and patches Babel/TypeScript to run Sucrase instead. It would be fragile, but it would let you try out Sucrase without needing to make any changes to your build config. It wouldn’t make sense for real use, but it could lower the barrier to entry.

Integrations

• Improve the current integrations (gulp, jest, require hook), e.g. providing better ways to specify options.
• Add a Webpack loader.
• Add a Rollup plugin.
• Add a node ES module loader hook so it’s possible to transform modules on-the-fly using native ES module support in node.
• Any other integrations that seem useful.
• See if there’s a nicer way to share the build system across projects and manage publishing and versions.

Performance

• Prototype a part of the project (probably the lexer) in AssemblyScript.
• Prototype a part of the project in Rust, both compiled to WebAssembly and run natively.
• See how practical it would be to write a custom WebAssembly code generator.
• Further simplify the parser by removing code that we know isn’t needed.
• Dumb down all of the code so that it’s more reasonable to run in WebAssembly (no classes or inheritance, no dynamic objects, etc.).
• Find ways to skip unnecessary parsing details, e.g. skipping all tokens in curly braces when parsing a type.
• Avoid backtracking in the parser, or find a way to make it more performant.
• Consider using lower-level features, e.g. ints instead of strings to identify tokens, and mutating an array rather than building a string using +.
• Consider simplifying the token format, e.g. making it so each type is just one full token.

[zerkalica]

• d.ts and flow declarations generation
• resolve import aliases. ts: baseUrl, paths from tsconfig.json support, flow: module.name_mapper from .flowconfig

[Raynos]

Allow Sucrase to read a tsconfig.json for its config (including which directories to compile and which to exclude, etc).

Implemented in https://github.com/alangpierce/sucrase/pull/509