bug(esm): TypeScript is not an ECMAScript superset post-ES2015

Bug Report

This bug report will show that TypeScript is no longer an ECMAScript subset as of ES2015 due to its refusal to support import specifier rewrites for contrived and misplaced reasons.

Preface

This issue is related to previously reported issues, but expands on them and clarifies the scope of the problem.

This team, mostly Andrew and Ryan, have repeatedly shut down these issues in the past. I intend to forward this issue to executives and ask that it not be closed as a kneejerk reaction, though I’m presuming that’s what will happen anyway and will have a text copy saved locally.

TS is designed to be an ES superset

TypeScript doubtless has many design goals, and I understand how hard it is to balance all of them. However, undeniably, its core design goal - what TypeScript exists to do - is stated in this repository’s description and also throughout the TS docs:

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

We should all agree that TS, first and foremost, is supposed to be an ECMAScript superset. There is no amount of mental gymnastics that gets us out of this. We will proceed with this in mind as we cover the scope of this issue and some of the technical excuses that have been offered to justify not addressing the problem.

What is a superset?

Originally a set theory concept, in terms of language it was best stated by @sepp2k in this StackOverflow answer:

Syntactically a language A is a subset of a language B if every program that is valid in language A is also valid in language B. Semantically it is a subset if it is a syntactic subset and each valid A program also exhibits the same behavior in language B.

To formalize this, Sebastian offers the following definition:

If P_A is the set of all valid programs in language A and P_B is the set of all valid programs in language B, then language A is a syntactic subset of language B exactly if P_A is a subset of P_B.

We will not need to address the semantic subset issue as TS fails the first test for arbitrarily many input programs¹, which we will get into.

For convenience, and to emphasize the point that our subset programs are actually emitted JS versions of the input TS program, we will refer to a program written in the superset language as “program T” and the corresponding output in the subset language as “program T'”.

Additionally, because TS output depends on your compiler configuration, we will consider TypeScript to pass the semantic test if there is some configuration for which you can transform a valid T into T'.

_{¹ This is something this team is already well aware of, and have simply chosen to ignore. In my judgment, the only possible explanation for violating the core purpose of this technology for such a large portion of its lifecycle is organizational deficiency, and not any practical technical reason.}

Is TypeScript a JS superset?

Sometimes. Let’s walk through some examples (with approximated and simplified, not literal, output):

1. ✅ Program T

   const a: string = "hello world!"
   console.log(a);
   

   ✅ Program T'

   const a = "hello world!"
   console.log(a);
   

   The input program is valid and the output program is valid. The runtime behavior of T and T' are identical. This example passes both the syntactic and semantic superset tests.

2. ✅ Program T (using relative extensionless TS import)

   import { a } from "./a"
   console.log(a);
   

   ❌ Program T' (TS-style imports left untransformed)

   import { a } from "./a"
   console.log(a);
   

   The input program is valid, but there is no configuration for which you will get a valid ECMAScript module output. The output program T' will throw according to the ES spec. This example fails the syntactic and semantic superset tests. Our perfectly valid TypeScript program cannot be transformed by the compiler into a valid ECMAScript program, and this is easily shown.

   While we can construct a different input program for which we will receive valid output (import { a } from "./a.js"), making that suggestion is proof positive that TS is no longer an ES superset, and there are infinitely many valid input programs which will not emit valid output. Nothing about this is subjective or within the realm of personal judgment.

Bearing this in mind, the last version of ES for which TypeScript was a superset would be ES5 (2009), before the introduction of ES modules.

Why does this matter?

It matters because our industry has struggled to adopt ESM for a variety of reasons, largely related to interop with downstream CJS consumers and also integration with TypeScript. It should be fairly obvious the damage that is done to the ecosystem when the TypeScript compiler very literally cannot generate valid ESM programs without special syntax.

The idea of having to use a special syntax to achieve a correct output program is inherently antithetical to the “TS is an ES superset” principle on which TypeScript is based. It takes a severe amount of technical dysfunction to turn a blind eye to this for years and years, and a serious amount of contempt for users and software quality to continually shut down user feedback on this issue.

Inadequacy of existing solutions and team response to user feedback

I have clarified several times that I am chalking this up to organizational dysfunction and attempting to kick it up the chain myself, as many many many users have raised this issue in isolation only to be shut down and/or directly gaslit by members of this team.

Let’s go over some of these issues. I will start with the closest suggestion to a fix currently under consideration, and then go through the rest in chronological order.

(!) 2020: allow voluntary .ts suffix for import paths (#37582) (by @timreichen)

I will start with this suggestion as it comes the closest to fixing the problem, while also offering a two-birds-one-stone benefit with regard to Deno, Bun, etc. support:

let import a from "path/to/a.ts" behave the same as import a from "path/to/a"

As specified, this partially solves our problem. Because import "./a.ts" could only ever refer to a TypeScript program, you cannot conceive of any input .ts import statement which will not correctly build to a .js import and execute equivalently at runtime.

However:

1. Because it is apparently “core design goal” that the imports never get rewritten, the most important part of this suggestion cannot be honored.

2. (Very important) This demands you rewrite your existing, valid import "./a" statements to ./a.ts, and your valid ./a imports still do not build to valid ES import statements.

Thus, however we might choose to emit the statements from input, we must either:

1. Resolve extensionless relative specifiers which are valid TS; OR
2. Deprecate extensionless relative specifiers in TypeScript, and make them invalid TS, to bring T and T' to parity.

The current approach by this team has literally just been to do nothing and deny that there is a conflict here. This conflict and its relationship to TS as an ES superset is the core focus of this issue.

2017: TypeScript and <script type="module"></script> (#13422) (by @cyrilletuzi)

Because the problem is, again, that TypeScript ESM input programs cannot build to valid output, naturally <script type="module"> goes right out the window.

Same pattern: User opens thread, years of comments and confusion, denialism from team members, summarily closed.

2017: Provide a way to add the ‘.js’ file extension to the end of module specifiers (#16577) (by @quantuminformation)

This issue looks like the others we will see: A user raising the issue that their valid TypeScript program cannot compile to a valid ES program, a large debate between users and various TS maintainers, and eventually the issue being closed, locked, and buried.

As usual, users point out that TS will act like their imports are valid while refusing to resolve them at compile-time, highlighting the tradeoff I explained. Relative extensionless specifiers must either be allowed and transformed, OR disallowed entirely. Doing both only violates the TS superset principle and confuses users because they depend on their output programs being isomorphic with their inputs.

Also, to generalize this issue a bit, I don’t think it’s actually about adding a .js extension, but resolving the module specifier to an actual path, whatever the extension is.

This comment has 50 likes. The parent issue has nearly 300. Minimizing comments from the team were met with a negative reaction as usual.

2020: Compiled JavaScript import is missing file extension (#40878) (by @jameshfisher)

This is an interaction, and a restatement of the same problem, which is effectively the perfect example of not just the problem and the common sense solution to it, but also the community’s investment in this being solved, and the textbook denialism by maintainers. The bot response announcing the closure as “working as intended” has 65 downvotes.

Mr. Fisher raises the issue that a program containing valid TypeScript imports will not emit a valid corresponding ECMAScript program. As we will see, this issue is met with approval and enthusiasm by people who found it trying to understand why their programs will not build (after several hours, they will presumably put two and two together and realize it’s just not possible). OP is then promptly sandbagged by members of this team and the issue is erroneously closed.

After the issue is posted, Ryan quickly responds to this post by minimizing OP’s complaint and telling him to kick rocks:

TypeScript doesn’t modify import paths as part of compilation - you should always write the path you want to appear in the emitted JS, and configure the project as necessary to make those paths resolve the way you want during compilation

Again, as we’ve covered, this is not how supersets work. Simply telling someone to rewrite every already-valid TS import statement into a different one which will compile should seem absurd on its face given what we’ve gone over. This comment received about 20 downvotes.

Mr. Fisher responds noting that he was shocked to find referring to TS source files with .js (an extremely confusing and hacky solution that is unbecoming of even a temporary workaround, let alone a permanent solution) actually does resolve his problem. He then lays out a few of the many reasons why this is confusing and hacky. His comment received 50 positive reactions.

Ryan responds with another heavily-downvoted dismissal of this issue:

TS never takes an existing valid JS construct and emits something different, so this is just the default behavior for all JS you could write. Module resolution tries to make all of this work “like you would expect”

You would expect a valid TS import to become a valid JS import because it’s supposed to be a superset. It is not a discussion or an issue of perspective; TS claims to be a superset, if it is, the valid imports would remain valid at runtime. Yet they do not, and we have highly experienced and talented engineers effectively lying about what is and is not expected runtime behavior.

@RevealedFrom clarifies this at further length, though it should be pretty obvious already why there are several problems with this, least of all being the superset issue:

@RyanCavanaugh Writing “./dep.js” doesn’t sound logical. The file dep.js does not exist in the Typescript universe. This approach requires the coder to know the exact complied output and be fully aware of the compiled environment. It’s like having to know the CIL outputted and modify it here and there in order to code in C# successfully. Isn’t the whole idea of Typescript to abstract away Javascript?

import { foo } from "./dep" is legitimate Typescript, and it provides the information for Typescript to resolve all that is needed to type check and make the code compile successfully. So, the compiled output should work. Typescript should not be generating syntactically incorrect Javascript.

IMHO, this issue should be a bug.

Ultimately, it is a bug, and we know it is a bug for the reasons we’ve covered. Ryan responds again with a minimizing and heavily-downvoted statement I would characterize as dishonest at best, and an outright lazy lie at worst:

The whole idea of TypeScript is to add static types on top of JavaScript, not to be a higher-level language that builds to JS.

We have enough background to know this is wrong, though we are starting to get insight into the executive dysfunction that is causing this issue. Core maintainers are making up what the purpose of this technology is as they go along, apparently now privately rejecting that TS is or ever was meant to be a JS superset.

A user in this thread (@Gin-Quin) had to actually implement a bundler to work around this, something I also had to do:

I will add that I indeed created a library called tsc-esm as a workaround for this bug, and even though it works quite fine every time I use it (ie 100% of the times I need to create a library written in Typescript) I feel I should not have to patch the output of the Typescript compiler like I do.

And another user (@djfm) also:

@silassare I think probably a thousand person, me included, have written similar scripts. This is just absurd.

And the last comment from this thread I will cover, and emphasize, is this one by Mr Felber (@PatrickFelber):

This whole discussion is absurd. Valid typescript obviously transpiles to invalid ES6 code. I do not understand in which world this can be classified as “works as intended” behavior. Adding “.js” to import statements which actually refer to “.ts” files as workaround is too wild for me. For my current project I have free choice, therefore I’m gonna switch over to CommonJS target module code. CommonJS works without file extensions. In case one can live with CommonJS target module, this is an option to get around this problem.

The only people who claim to not see the problem are members of this team. Note that this user’s only choice was to give up and stay on CJS - this is what I am referring to when I say that this is likely the most major blockade to ESM adoption under Why does this matter?.

2020: TypeScript cannot emit valid ES modules due to file extension issue (#42151) (by me)

This issue was opened by me to clarify that the compiler cannot emit valid ES modules. It received positive feedback from users, but was naturally accompanied by denialism from Ryan and eventually closed and locked. A comment by @richardkazuomiller points out the ridiculousness of the justification offered for not fixing this issue (“we can’t rewrite import specifiers”):

Exactly, could someone explain like I’m 5 why it’s OK for TypeScript to convert import … from ‘./file’ to const … = require(‘./file’) changing ./file to ./file.js is not OK?

2021: One year later: TypeScript still cannot emit valid ES modules (#47270) (by me)

This is a rephrasing of the original issue, posted so as to remind this team that this problem has not been resolved. The single response from Ryan was amusing given the context of this issue:

We’ve explained the rationale behind not modifying import paths on emit many times, and don’t see value in rehashing that discussion since it relates to a core design goal. […] Please file something concrete in the future.

You must decide to prioritize a TypeScript “core design goal”, but you can only choose one:

1. Never rewriting import specifiers, even though the compiler does this for regular import statements
2. Ensuring TS is an ES superset like it says it is throughout its documentation

Apparently this is not an obvious answer to TS maintainers, though to users it apparently is. Furthermore this issue is an attempt to follow-up explicitly on the second half of Ryan’s request to file something concrete going forward, though I imagine this issue will also be summarily buried as this team appears to have only one tool in its toolbox, namely denial for the sake of convenience.

2020: Appending .js extension on relative import statements during Typescript compilation (ES6 modules) (StackOverflow)

Same situation we’ve seen: User has a valid input program, can’t get a valid output program. The answerer prudently notes:

If the compiler simply generated extension-less output files it would also solve the issue. But, would that also somehow violate the design principle regarding URI rewrites? Certainly, in that case there could exist other design principles to defend the position! But wouldn’t such stubbornness only help to further validate the adamancy or ignorance of the TS team on this issue?

2022: "module": "node16" should support extension rewriting (#49083) (by @arendjr)

This is the most recent issue that covers this problem. Mr. van Beelen is certainly a lot more generous and forgiving with his wording than I am, though he is equally rewarded with stark denialism from Ryan.

We don’t rewrite target-legal JavaScript code to be different JavaScript code from what it started with, no matter the configuration.

We won’t even get into the fact that import "./a" is not an ES2015 “target-legal” import statement, as it’s just an egregiously incompetent claim. It received about 60 downvotes.

Mr. van Beelen left several more long, thoughtful comments, generating over 100 positive responses, explaining much of the same things that I have covered here, that valid TS programs should become valid JS programs, though he did so with much more forgiving phrasing. I recommend everyone read his comments, and I found them very considerate - likely impractically so.

This is another thread that contains hundreds of comments, hundreds of positive reactions for the sensible position taken by users, and hundreds of negative reactions for this team’s excuses. It is impossible to read it all, but it is worth skimming over and noticing that it, like every other issue raised regarding TS’s inability to build valid output programs, was summarily closed for the sake of maintainer sanity to the detriment of several million codebases relying on TS.

2022: Add Compileroption to add Extensions to relative imports (#47436) (by @jogibear9988)

A PR to add non-breaking, opt-in compilerOption for adding .js extension to extensionless relative TS imports, which received about 30 positive reactions. Of course, it was summarily closed without a real review. No amount of attempts to contribute by the community have been well-received by this team.

🔎 Search Terms

• superset
• ESM
• imports

🕗 Version & Regression Information

TypeScript was initially released in 2012, and it has not been an ES superset since the release of ES2015, which require imports to have file extensions if they are not named modules.

⏯ Playground Link

Cannot repro in playground, requires multiple entry points. Minimum repro: https://replit.com/@ctjlewis/ts-esm-superset

Throws ERR_MODULE_NOT_FOUND because ./a is not a valid ES import specifier.

💻 Code

N/A, see repro and notes in this issue.

🙁 Actual behavior

The valid TS-ESM program T builds to an invalid program T'.

🙂 Expected behavior

The valid TS-ESM program T should build to a valid program T'. Valid TS imports should always compile to valid ES imports for the given target.

[fatcerberus]

I stopped reading here:

The output program T' will throw according to the ES spec.

The ECMAScript specification says nothing about the meaning of module specifiers. Module resolution is the host’s responsibility and is therefore implementation-defined. If the TS compiler can resolve an import but your runtime can’t, that just means your project is incorrectly configured for the target runtime environment and doesn’t say anything about ES compliance.

If you want to argue this behavior is suboptimal, that’s fine, but you can’t use spec compliance as a bludgeon here.

[ctjlewis]

Stockholm Syndrome ITT.