Remove runtime renderer, still support MDXProvider

[wooorm]

In short

What

This PR moves most of the runtime to the compile time.

Note

This issue has nothing to do with @mdx-js/runtime. It’s about
@mdx-js/mdx being compile time, and moving most work there, from the
“runtimes” @mdx-js/react, @mdx-js/preact, @mdx-js/vue.

Why

Most of the runtime is undocumented features that allow amazing things,
but those are in my opinion too magical, more powerful than needed,
complex to reason about, and again: undocumented.
These features are added by overwriting an actual renderer (such as
react, preact, or vue). Doing so makes it hard to combine MDX with for
example Emotion or theme-ui, to opt into a new JSX transform when React
introduces one, to support other hyperscripts, or to add features such
as members (<Foo.Bar />). Removing these runtime features does what
MDX says in the readme: “🔥 Blazingly blazing fast: MDX has no
runtime […]”

Downsides

This does remove the ability to overwrite anything at runtime. This
brings back the project to what is documented: users can still
overwrite markdown things (e.g., blockquotes) to become components and
pass components in at runtime without importing them. And it does still
allow undocumented parent-child combos (blockquote.p).

Changelog

• Remove runtime renderers (createElements hijacking) from
  @mdx-js/react, @mdx-js/preact, @mdx-js/vue
• Add jsxRuntime option to switch to the modern automatic JSX runtime
• Add jsxImportSource option to switch to a modern non-React JSX
  runtime
• Add pragma option to define a classic JSX pragma
• Add pragmaFrag option to define a classic JSX fragment
• Add mdxProviderImportSource option to load an optional runtime
  provider
• Add tests for automatic React JSX runtime
• Add tests for @mdx-js/mdx combined with emotion
• Add support and test members as “tag names” of elements
• Add support and test qualified names (namespaces) as “tag names” of
  elements
• Add tests for parent-child combos
• Add tests to assert explicit (inline) components precede over
  provided/given components
• Add tests for mdxFragment: false (runtime renderers w/o fragment
  support)
• Fix and test double quotes in attribute values

In long

This PR removes the runtime renderers and related things such as the
mdxType and parentName props while keeping the MDXProvider in
tact.

This improves runtime performance, because all that runs at runtime is
plain vanilla React/preact/vue code.

This reduces the surface of the MDX API while being identical to what
is documented and hence to user expectations (except perhaps to some
power users).

This also makes it easier to support other renderers without having to
maintain projects like @mdx-js/react, @mdx-js/preact, @mdx-js/vue:
anything that can be used as a JSX pragma (including the automatic
runtime)
is now supported.
A related benefit is that it’s easier to integrate with
emotion
(including through theme-ui) and similar projects which also
overwrite the renderer: as it’s not possible to have two runtimes, they
were hard to combine; because with this PR MDX is no longer a renderer,
there’s no conflict anymore.

How?

This is done by the compile time (@mdx-js/mdx) knowing about an
(optional) runtime for an MDXProvider (such as @mdx-js/react,
@mdx-js/preact). Importantly, it’s not required for other
hyperscript interfaces to have a provider: MDXContent exported from
a compiled MDX file also accepts components (it already did), and Vue
comes with component passing out of the box.

What did the runtime do?

In short, the runtime looked like this:

function mdx(thing, props, ...children) {
  const overwrites = getOverwritesSomeWay()
  return React.createElement(overwrites[props.mdxType] || thing, props, ...children)
}

And we had a compile time, which added that mdxType prop. So:

<Youtube />

Became:

const Youtube = () => throw new Error('Youtube is not loaded!')

<Youtube mdxType="Youtube" />

Which in plain JS looks like:

const Youtube = () => throw new Error('Youtube is not loaded!')

React.createElement(Youtube, {mdxType: 'Youtube'})

Instead, this now compiles to:

const {Youtube} = Object.assign({Youtube: () => throw new Error('Youtube is not loaded!')}, getOverwritesSomeWay())

React.createElement(Youtube)

The previous example shows what is sometimes called a “shortcode”: a
way to inject components as identifiers into the MDX file, which was
introduced in MDX 1

A different use case for the runtime was overwriting “defaults”. This
is documented on the website as the “Table of
components”. This MDX:

Hello, *world*!

Became:

<p mdxType="p">Hello, <em mdxType="em">world</em>!</p>

This now compiles to:

const overwrites = Object.assign({p: 'p', em: 'em'}, getOverwritesSomeWay())

<overwrites.p>Hello, <overwrites.em>world</overwrites.em>!</overwrites.p>

The not so pretty parts / undocumented features of the previous runtime

Overwriting explicit components

This MDX:

export const Video = () => <Vimeo />

<Video />

Used like so:

<MDXProvider components={{Video: () => <Youtube />}}>
  <Content />
</MDXProvider>

Would result in a Youtube component being rendered. It no longer
does. I see the previous behavior as a bug and hence this as a fix.

Overwriting MDXLayout

A subset of the above point is that:

export default props => <main {...props} />

x

Used like so:

<MDXProvider components={{wrapper: props => <article {...props} />}}>
  <Content />
</MDXProvider>

Would result in an article instead of the explicit main. It no
longer does. I see the previous behavior as a bug and hence this as a
fix.

Sandboxing

(#821)

## Hello

<h2>World</h2>

Used like so:

<MDXProvider components={{h2: () => <SomethingElse />}}>
  <Content />
</MDXProvider>

Would result in a SomethingElse for both. This PR does not change
that. But it could more easily be changed if we want to, because at
compile time we know whether something was a tag or not.

Overwriting anything

An undocumented feature of the current MDX runtime renderer is that
it’s possible to overwrite anything:

<span />

Used like so:

<MDXProvider components={{span: props => <b>{props.children}</b>}}>
  <Content />
</MDXProvider>

Would overwrite to become bold, even though it’s not documented
anywhere. This PR changes that: only allowed markdown “tag names” can
be changed (p, li, ...). This list could be expanded.

Parents

Another undocumented feature is that parent–child combos can be
overwritten. A li in an ol can be treated differently from one in
an ul by passing 'ol.li': () => <SomethingElse />.

This PR no longer lets users “nest” arbitrary parent–child combos
except for ol.li, ul.li, and blockquote.p. This list could
be expanded.

Members

It was not possible to use members (<foo.bar />, <Foo.bar.baz />,
#953) and supporting it previously
would be complex. This PR adds support for them.

Scope pollution

Previously, mdxType and parentName attributes were added to all
elements. And a components prop was accepted on all elements to
change the provider. These are no longer passed and no longer accepted.
Lastly, components, props were in scope for all JSX tags defined in
the “markdown” section (not the import/exports) of each document.

This adds identifiers to the scope prefixed with double underscores:
__provideComponents, __components, and __props.

Benchmark

Input

A single 1mb MDX file, about 20k lines and 135k words (basically 3
books). Heavy on the “markdown”, few tags, no import/exports.
322kb gzipped.

Build (ms)

• v1: 2895.122856
• 2.0.0-next.8: 3187.4684129999996
• main: 4058.917152000001
• this pr: 4066.642403

Output (CJS, not bundled, JSX compiled away)

• v1: raw: 1.5mb, gzip: 348kb
• 2.0.0-next.8: raw: 1.4mb, gzip: 347kb
• main: raw: 1.3mb, gzip: 342kb
• this pr: raw: 1.8mb, gzip: 353kb
• this pr, automatic runtime: raw: 1.7mb, gzip: 355kb

Run (ms)

• v1: 321.761208
• 2.0.0-next.8: 321.79749599999997
• main: 162.412757
• this pr: 107.28038599999996
• this pr, automatic runtime: 123.73588899999999

Result

This PR is much faster on giant markdown-esque documents during runtime.
The win over the current main branch is 34%, the win over the last
beta and v1 is 66%.

For output size, the raw value increases with this PR, which is because
the output is now /*#__PURE__*/React.createElement(__components.span…)
or /*#__PURE__*/_jsx(__components.span…), instead of mdx("span", {mdxType: "span"…}). The change is more repetition, as can be seen by
the roughly same gzip sizes.

That the build time of main and this PR is slower than v1 and the
last beta does surprise me a lot. I benchmarked earlier with 1000 small
simple MDX files, totalling 1mb, where the results were the
inverse. So
it looks like we have a problem with giant files. Still, this PR has no
effect on build time performance, because the results are the same as
currently on main.

TL;DR

This PR makes MDX faster, adds support for the modern automatic JSX
runtime, and makes it easier to combine with Emotion and similar
projects.

---

Some of what this PR does has been discussed over the years:

Related-to: GH-166.
Related-to: GH-197.
Related-to: GH-466 (very similar).
Related-to: GH-714.
Related-to: GH-938.
Related-to: GH-1327.

This PR solves some of the items outlined in these issues:

Related-to: GH-1152.
Related-to: #1014 (comment).

This PR solves:

Closes GH-591.
Closes GH-638.
Closes GH-785.
Closes GH-953.
Closes GH-1084.
Closes GH-1385.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/mdx/mdx/m49xi7cvg
✅ Preview: Failed

[ChristopherBiscardi]

as detailed in #1385 this is unmergeable as it removes functionality core to mdx

[wooorm]

Such as?

[ChristopherBiscardi]

Such as?

I'm getting extremely tired of your gaslighting. you clearly explain that there are unnecessary breaking changes in this PR description, and we've already discussed them in the other issue I mentioned.

[ChristianMurphy]

we've already discussed them in the other issue I mentioned

We have discussed, and you insights and suggestions from that issue are (intended to be) addressed in this PR.
See #1385 (comment)

I'm getting extremely tired of your gaslighting.

Respectfully, I don't think a PR, which is opened with the goal of addressing comments you made is "gaslighting".

[wooorm]

landed!