Alternative to createMdxAstCompiler without mdxAstToMdxHast

I want an export like createMdxAstCompiler but I also want to preserve the MDX syntax tree, instead of getting a HTML syntax tree back.

This is for a Vite plugin that uses the local @mdx-js/mdx, so importing remark-parse, remark-mdx, etc is not viable.

[ChristopherBiscardi]

@ChristianMurphy it’s because without the remark AST it’s harder to find the import nodes, or combine the ASTs from two different MDX files (prebuilt transclusion, etc). You can see where it’s happening here in the linked plugin. (Also see https://github.com/mdx-js/mdx/issues/1423 for a similar, but not exactly the same use case).

I also have a use case for this in Sector, the block editor I’m building on top of MDX and the workaround I use is the same as the one posted here (copy/paste the createMdxAstCompiler bits without the hast plugin) and I’ve talked to @johno about including this behavior as if Sector needs it then others will as well to build more sophisticated tools on top of MDX.

@ChristianMurphy In the future, if you don’t understand the use case please avoid linking to the xy problem question. You will get better responses from users if you don’t link to 10 year old stackexchange questions.

[ChristianMurphy]

@aleclarson this comes back to the XY problem. What I am hearing from you and @ChristopherBiscardi is that it is not clear how to access and work with the MDAST+MDX AST. That is valid and valuable feedback, thank you both!

You’ve both suggested a solution, adding a new API to access the Markdown+MDX AST, inside @mdx-js/mdx. Which is also appreciated, thank you again!

What is missing, is that this is not the only solution. There are others, including:

• Publishing a new utility package in the @mdx-js name space
• Having community members publish a package with the utility method outside the @mdx-js namespace
• Improving the documentation and guides with more information on the Markdown+MDX AST and how to work with it

Which leads to which approach should be taken?

Adding a new method to @mdx-js/mdx would offer a way to get the AST. But would also increase the API surface, adding more to document and maintain, in addition, adding a second method similar to createMdxAstCompiler, which could confuse adopters, with care needed to understand which of several closely related APIs actually meets their needs.

An alternative, improving the documentation and guides to add more information on Markdown+MDX AST and how to work with it with the existing API, seems to address the core concern of making it more approachable and easier for new adopters to consider leveraging it, without increasing API surface or adding a potentially confusing method. This is the approach I would currently favor.

That said, the approaches are not mutually exclusive, a method could be added and the documentation improved.

“every possible combination” is hyperbole. The only case I want to be supported out-of-the-box is “everything MDX does by default, excluding HTML-specific stuff”.

Not entirely. MDX provides a method for accessing the full AST transform pipeline.

We could add one to access the Markdown+MDX AST specifically. We could also add one to take the Markdown+MDX AST and transform it to HAST+MDX. We could add ones which don’t remove spaces or don’t squeeze paragraphs. All of the above would be valuable to someone, and for good reason too. But how many belong as utilities maintained in @mdx-js/mdx? Why or why not?

That seems like a broader discussion, which should include @mdx-js/core and @mdx-js/maintainers.

Personally I see these as better suited for documentation, than an ever increasing core API. But am open and interested in reasoning that sees otherwise.

Thoughts? 💭