API to Compile to fragment instead of component

Subject of the feature

This is either a question or feature request. I’d like to be able to take a “partial” of mdx and render to just a jsx fragment, instead of a full component with a layout, for rendering into an existing mdx page. In other words it’d be nice to get just the return value of the MDXContent component.

Problem

Not a problem, but the use case is extracting md/mdx comments from code, via tools like react-docgen or TS typedoc and render them into other mdx pages. E.g. enhancing hand written MDX doc files with auto generated docs. Being able to output fragments allows “stitching” a few compiled fragments into a single file.

Expected behavior

You could imagine something like the following for rendering out importable metadata in, e.g. Docusaurus.

let imports = []
let exports = []

propsMetadata.forEach((prop) => {
  const result = mdx.compileTofragment(prop.description)
  prop.mdx = result.fragment
  imports.push(...result.imports)
  exports.push(...result.exports)
})

writeFile(`
  ${imports.join(';\n')}

  ${exports.join(';\n')}

  export default ${stringifyToJs(propsMetadata, null, 2)
`)

The idea here is that the jsx fragments could be rendered where ever (in the context of another MDX page). I actually find the need for imports/exports less important since you probably aren’t likely to be using them in the context of a “mdx” fragment, so it’s a bit weird semantically. I don’t think it’s necessary to support but figured I could show a possible API to be comprehensive.

Alternatives

The current API can be used to write each block out to its own file and import it manually, but it’s a bit clunky and gives unwanted behavior like wrapping every block in a layout as if it was a standalone page. Perhaps this is already possible and i’m just missing an API!

[wooorm]

This is now possible with the latest RC for MDX 2. See https://v2.mdxjs.com. The heavy lifting can be done by setting options.outputFormat to 'function-body' to compile to, well, a function body, rather than a whole program. You can wrap those function bodies in IIFEs:

import {compile} from '@mdx-js/mdx'

main(['# some mdx', 'Some more {1 + 1}', '> Ta da!', 'export const a = "b"'])

async function main(descriptions) {
  const file =
    'export default function createFragments(jsxRuntime) { return [' +
    (
      await Promise.all(
        descriptions.map(async (d) => {
          return (
            '(function () {' +
            (await compile(d, {outputFormat: 'function-body'})) +
            '})(jsxRuntime)'
          )
        })
      )
    ).join(',') +
    ']}'

  console.log(file)
}

The above prints something along these lines (formatted and abbreviated):

export default function createFragments(jsxRuntime) {
  return [
    (function () {
      const {Fragment: _Fragment, jsx: _jsx} = arguments[0]
      function MDXContent(props = {}) {
        const _components = Object.assign({h1: 'h1'}, props.components)
        const {wrapper: MDXLayout} = _components
        const _content = _jsx(_Fragment, {children: _jsx(_components.h1, {children: 'some mdx'})})
        return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {children: _content})) : _content}
      }
      return {default: MDXContent}
    })(jsxRuntime),
    /* … */
  ]
}

Of course, you might want slightly different wrapper code and maybe use an object mapping names to fragments. Note that exports are supported and imports could be supported with options.useDynamicImport (and async IIFEs). Now, assuming you wrote that to the file system as fragments.js, and imported it somewhere where React/Preact/Emotion/etc is available, it could be used like so:

import * as jsxRuntime from 'react/jsx-runtime'
import createFragments from './fragments.js'

console.log(createFragments(jsxRuntime))

Which prints:

[
  { default: [Function: MDXContent] },
  { default: [Function: MDXContent] },
  { default: [Function: MDXContent] },
  { a: 'b', default: [Function: MDXContent] }
]

[ChristopherBiscardi]

@jquense If you copy/paste a bit of code and take line 19 out: https://github.com/mdx-js/mdx/blob/c1269258e14150621f4c7aacefe96c564b518547/packages/mdx/index.js#L19 you can parse to the remark ast, which you can then merge at will with other remark asts.

I’ve done this sort of thing before and while it feels a bit off to be copy/pasting code like that, that particular file doesn’t change often, so should be low-effort if you want to pursue this route.

@johno we’ve chatted about ast-level transclusion before, do you think exposing more of the compiler-level APIs makes sense? An MDX editor I’m working on, for example, also needs to go from mdx string to remark-mdx ast and back. unified tends to make it kinda of hard to work with intermediate objects outside of it’s plugin system.

here’s a file I’m using for the purpose

import footnotes from "remark-footnotes";
import remarkMdx from "remark-mdx";
import remarkMdxJs from "remark-mdxjs";
import squeeze from "remark-squeeze-paragraphs";
import toMDAST from "remark-parse";
import unified from "unified";
import remarkStringify from "remark-stringify";
import json5 from "json5";
import visit from "unist-util-visit";

export function pluckMeta(value) {
  const re = new RegExp(`^export const meta = `);
  let meta = {};
  if (value.startsWith(`export const meta = `)) {
    const obj = value.replace(re, "").replace(/;\s*$/, "");
    meta = json5.parse(obj);
  }
  return meta;
}
// a remark plugin that plucks MDX exports and parses then with json5
export function remarkPluckMeta({ exportNames }) {
  return (tree, file) => {
    file.data.exports = {};
    exportNames.forEach((exportName) => {
      const re = new RegExp(`^export const ${exportName} = `);
      visit(tree, "export", (ast) => {
        if (ast.value.startsWith(`export const ${exportName} = `)) {
          const obj = ast.value.replace(re, "").replace(/;\s*$/, "");
          file.data.exports[exportName] = json5.parse(obj);
        }
      });
    });
    return tree;
  };
}

/// Stringify mdxast from nodes
export const processor = unified()
  .use(remarkStringify, {
    bullet: "*",
    fence: "`",
    fences: true,
    incrementListMarker: false,
  })

  .use(remarkMdx)
  .use(remarkMdxJs);

// Parse mdxast to nodes
const DEFAULT_OPTIONS = {
  remarkPlugins: [],
  rehypePlugins: [],
};

function createMdxAstCompiler(options) {
  const plugins = options.remarkPlugins;

  const fn = unified()
    .use(toMDAST, options)
    .use(remarkMdx, options)
    .use(remarkMdxJs, options)
    .use(footnotes, options)
    .use(squeeze, options);

  plugins.forEach((plugin) => {
    // Handle [plugin, pluginOptions] syntax
    if (Array.isArray(plugin) && plugin.length > 1) {
      fn.use(plugin[0], plugin[1]);
    } else {
      fn.use(plugin);
    }
  });

  return fn;
}

function createCompiler(options = {}) {
  const opts = Object.assign({}, DEFAULT_OPTIONS, options);
  const compiler = createMdxAstCompiler(opts);
  return compiler;
}

export const parse = createCompiler().parse;
export const stringify = processor.stringify;