Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Vue, test and document rest #1338

Merged
merged 6 commits into from
Nov 19, 2020
Merged

Fix Vue, test and document rest #1338

merged 6 commits into from
Nov 19, 2020

Conversation

wooorm
Copy link
Member

@wooorm wooorm commented Nov 16, 2020

/cc @codebender828 for the Vue part, @ChristopherBiscardi for the Preact part, any anyone else who wants to take a look (it’s a bit noisy due to the readme changes)

Hi folks! I added some more docs to readmes and added more tests. I couldn’t get the repo to work (latest yarn seems broken), so I locked that. Most of the work was on the tests, trying to get to 100% coverage. I found a lot of tests that were theoretical (e.g., didn’t evaluate a loaded MDX file). We’re now evaluating more MDX code. Specifically, it seems Vue was (mostly) broken.

And: what’s the withMDXComponents in (p)react for? It was never tested, and is only used in an old example in the docs.

Changes

  • Update readmes
  • Fix code style
  • Add 100% coverage to renderers, babel plugins, loaders, runtime
  • Add todos inline for potentially broken things
  • Fix Vue to have tests, support properties, support Vue components
  • Lock Yarn because latest is broken
  • Soft deprecate (by adding private: true) @mdx-js/util, @mdx-js/test-util, babel-plugin-html-attributes-to-jsx: their contents weren’t used or not reused
  • Set up tests in each package, to allow for different test methods (e.g., vue)
  • Largely not touching code (except for Vue only stylistic changes)

* Update readmes
* Fix code style
* Add 100% coverage to renderers, babel plugins, loaders, runtime
* Add todos inline for potentially broken things
* Fix Vue to have tests, support properties, support Vue components
* Lock Yarn because latest is broken
* Soft deprecate `@mdx-js/util`, `@mdx-js/test-util`,
  `babel-plugin-html-attributes-to-jsx`: their contents weren’t used or not
  reused
* Set up tests in each package, to allow for different test methods (e.g.,
  vue)
* Largely not touching code
@vercel
Copy link

vercel bot commented Nov 16, 2020

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/mdx/mdx/ktizvtj21
✅ Preview: Failed

[Deployment for cce24a6 failed]

Copy link
Member

@ChristianMurphy ChristianMurphy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting this started @wooorm!

.yarnrc Outdated Show resolved Hide resolved
packages/babel-plugin-extract-export-names/index.js Outdated Show resolved Hide resolved
Comment on lines +60 to +67
// To do: is this correct?
test('should capture renames', () => {
expect(
transform(
'var name1 = "a", name2 = "b"; export {name1 as foo, name2 as bar}'
)
).toEqual(['name1', 'name2'])
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.
The names should be aliased as foo and bar

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this package is used more for figuring out which local values are also exported (not caring about as what they’re exported).

const importExportNames = importNames.concat(exportNames)
const fakedModulesForGlobalScope =
`const makeShortcode = name => function MDXDefaultShortcode(props) {
console.warn("Component " + name + " was not imported, exported, or provided by MDXProvider as global scope")
return <div {...props}/>
};
` +
uniq(jsxNames)
.filter(name => !importExportNames.includes(name))
.map(name => `const ${name} = makeShortcode("${name}");`)
.join('\n')

I’m not 100% whether doing it like this, and at build time, is useful? I’d say checking if this was nully at runtime would be better?

const Component =
components[`${parentName}.${type}`] ||
components[type] ||
DEFAULTS[type] ||
originalType

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I’ll just leave {import,export}-names and see when I start on mdx-js/mdx whether they can be deprecated

packages/loader/index.js Show resolved Hide resolved
packages/preact/test/test.js Show resolved Hide resolved
.eslintignore Outdated Show resolved Hide resolved
packages/vue/src/create-element.js Show resolved Hide resolved
packages/vue/src/create-element.js Show resolved Hide resolved
packages/vue/src/create-element.js Outdated Show resolved Hide resolved
packages/vue-loader/index.js Outdated Show resolved Hide resolved
@vercel vercel bot temporarily deployed to Preview November 17, 2020 08:15 Inactive
@ChristianMurphy

This comment has been minimized.

.eslintignore Outdated Show resolved Hide resolved
.yarnrc Outdated Show resolved Hide resolved
* Stop locking dev-dependencies
* Move several dev-dependencies from the root `package.json` to the places
  that depend on them
* Remove unused dev-dependencies
* Remove unused `yarn.lock`s in non-root
* Use `microbundle` for preact, react, and vue
* Fix installing, building, formatting, and testing
@vercel vercel bot temporarily deployed to Preview November 18, 2020 12:32 Inactive
@vercel vercel bot temporarily deployed to Preview November 18, 2020 13:07 Inactive
@vercel vercel bot temporarily deployed to Preview November 18, 2020 13:13 Inactive
@wooorm
Copy link
Member Author

wooorm commented Nov 18, 2020

Well, actions is a bit slow but it’s somewhat working.

@ChristianMurphy this error might be something for you: https://github.com/mdx-js/mdx/pull/1338/checks?check_run_id=1418253147#step:5:135

And if someone knows why lerna run test or so, while we have "npmClient": "yarn" in lerna.json, yields:

npm WARN lifecycle The node binary used for scripts is /tmp/yarn--1605705373202-0.3365631855975586/node but npm is using /opt/hostedtoolcache/node/10.23.0/x64/bin/node itself. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.

I’m all ears 🤷‍♂️

.eslintrc.yml Show resolved Hide resolved
.github/workflows/lint.yml Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
packages/loader/index.js Show resolved Hide resolved
@ChristianMurphy
Copy link
Member

npm WARN lifecycle The node binary used for scripts is /tmp/yarn--1605705373202-0.3365631855975586/node but npm is using /opt/hostedtoolcache/node/10.23.0/x64/bin/node itself. Use the `--scripts-prepend-node-path` option to include the path for the node binary npm was executed with.

Not sure, and not too worried, since it's a warning not an error.
Yarn has a mix of actual warnings and yarn opinions logged as warnings, it can be hard to tell them apart, my guess is this is an opinion as a warning.
Ideally I'd like to see the project move to npm 7, and drop yarn entirely, but that's another discussion for another PR.


More concerning is:

Error: Cannot find module '/home/runner/.dts/typescript-installs/4.0/node_modules/typescript'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at testDependencies (/home/runner/work/mdx/mdx/node_modules/dtslint/bin/lint.js:64:16)
    at Object.<anonymous> (/home/runner/work/mdx/mdx/node_modules/dtslint/bin/lint.js:26:28)
    at Generator.next (<anonymous>)
    at /home/runner/work/mdx/mdx/node_modules/dtslint/bin/lint.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/home/runner/work/mdx/mdx/node_modules/dtslint/bin/lint.js:4:12

This might be a network connectivity issue, or it may be an issue with the cache 🤔

@wooorm
Copy link
Member Author

wooorm commented Nov 18, 2020

A rerun showed different network timeouts. Maybe what we’re installing is just too much for GH to handle 😅

@vercel vercel bot temporarily deployed to Preview November 19, 2020 13:11 Inactive
@ChristianMurphy
Copy link
Member

Thanks @wooorm, looks like github actions is getting happier.
The Vercel build seems to be running into a new issue:


06:14:49.259 | Generating JavaScript bundles failed
-- | --
06:14:49.260 | Cannot find module '@mdx-js/mdx/util'
06:14:49.260 | Require stack:
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby-plugin-mdx/loaders/mdx-loader.js
06:14:49.260 | - /vercel/workpath0/node_modules/loader-runner/lib/loadLoader.js
06:14:49.260 | - /vercel/workpath0/node_modules/loader-runner/lib/LoaderRunner.js
06:14:49.260 | - /vercel/workpath0/node_modules/webpack/lib/NormalModule.js
06:14:49.260 | - /vercel/workpath0/node_modules/webpack/lib/NormalModuleFactory.js
06:14:49.260 | - /vercel/workpath0/node_modules/webpack/lib/Compiler.js
06:14:49.260 | - /vercel/workpath0/node_modules/webpack/lib/webpack.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby/dist/commands/build-html.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby/dist/commands/build.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby-cli/lib/create-cli.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby-cli/lib/index.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby/dist/bin/gatsby.js
06:14:49.260 | - /vercel/workpath0/node_modules/gatsby/cli.js
06:14:49.260 | File: packages/gatsby-theme-mdx/src/pages/playground.mdx

https://vercel.com/mdx/mdx/lvagnsynz

@wooorm
Copy link
Member Author

wooorm commented Nov 19, 2020

I’ll see what I can do. I don‘t know Gatsby. FWIW the tests have been failing for a while now https://github.com/mdx-js/mdx/commits/next, I’d rather start moving to smaller PRs

@wooorm wooorm merged commit e7e9d46 into next Nov 19, 2020
@wooorm wooorm deleted the docs branch November 19, 2020 15:33
@wooorm wooorm mentioned this pull request Nov 22, 2020
@wooorm wooorm added 📚 area/docs This affects documentation 🕸️ area/tests This affects tests 💪 phase/solved Post is done 🦋 type/enhancement This is great to have 🧑 semver/major This is a change labels Nov 22, 2020
wooorm added a commit that referenced this pull request Dec 20, 2020
This removes the last three custom Babel plugins we had and replaces
them with estree versions.
Furthermore, it removes `@babel/generator`.

For the plugins, we were only looking at ESM import/exports, but right
now we’re delegating work to `periscopic` to look at which things are
defined in the top-level scope.
It’s a bit more complex, but this matches better with intentions,
fixes some bugs, and prepares for a potential future where other ES
constructs are allowed, so all in all should be a nice improvement.

For serializing, we’re switching to `astring`, and handling JSX for now
internally (could be externalized later).
`astring` seems fast and is incredibly small, but is not very popular.
We might perhaps see bugs is serialization in the future because of that,
but all our tests seem fine, so I’m not too worried about that.

Estree remains a somewhat fragmented ecosystem, such as that the tree
walkers in `periscopic` and `astring` are different, so we might also
consider writing our own serializer in the future.
Or, when we implement Babel’s React JSX transform ourselves, could switch
to another generator, or at least drop the JSX serialization code here.

Because of these changes, we can drop `@babel/core` and
`@babel/generator` from `@mdx-js/mdx`, which drops the bundle size of
from 349kb to 111kb.
That’s 68%.
Pretty nice.
This should improve downloading and parsing time of bundles
significantly.
Of course, we currently still have JSX in the output, so folks will
have to resort to Babel (or `buble-jsx-only`) in another step.

For performance, v2 (micromark) was already an improvement over v1.
On 1000 simple files totalling about 1mb of MDX:

* v1: 3739ms
* v2: 2734ms (26% faster)
* v2 (w/o babel): 1392ms (63% faster).

Of course, this all really depends on what type of stuff is in your MDX.
But it looks pretty sweet!

✨

Related to GH-1046.
Related to GH-1152.
Related to GH-1338.
Closes GH-704.
Closes GH-1384.
@wooorm wooorm mentioned this pull request Dec 20, 2020
johno pushed a commit that referenced this pull request Dec 20, 2020
This removes the last three custom Babel plugins we had and replaces
them with estree versions.
Furthermore, it removes `@babel/generator`.

For the plugins, we were only looking at ESM import/exports, but right
now we’re delegating work to `periscopic` to look at which things are
defined in the top-level scope.
It’s a bit more complex, but this matches better with intentions,
fixes some bugs, and prepares for a potential future where other ES
constructs are allowed, so all in all should be a nice improvement.

For serializing, we’re switching to `astring`, and handling JSX for now
internally (could be externalized later).
`astring` seems fast and is incredibly small, but is not very popular.
We might perhaps see bugs is serialization in the future because of that,
but all our tests seem fine, so I’m not too worried about that.

Estree remains a somewhat fragmented ecosystem, such as that the tree
walkers in `periscopic` and `astring` are different, so we might also
consider writing our own serializer in the future.
Or, when we implement Babel’s React JSX transform ourselves, could switch
to another generator, or at least drop the JSX serialization code here.

Because of these changes, we can drop `@babel/core` and
`@babel/generator` from `@mdx-js/mdx`, which drops the bundle size of
from 349kb to 111kb.
That’s 68%.
Pretty nice.
This should improve downloading and parsing time of bundles
significantly.
Of course, we currently still have JSX in the output, so folks will
have to resort to Babel (or `buble-jsx-only`) in another step.

For performance, v2 (micromark) was already an improvement over v1.
On 1000 simple files totalling about 1mb of MDX:

* v1: 3739ms
* v2: 2734ms (26% faster)
* v2 (w/o babel): 1392ms (63% faster).

Of course, this all really depends on what type of stuff is in your MDX.
But it looks pretty sweet!

✨

Related to GH-1046.
Related to GH-1152.
Related to GH-1338.
Closes GH-704.
Closes GH-1384.
@wooorm wooorm mentioned this pull request Dec 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📚 area/docs This affects documentation 🕸️ area/tests This affects tests 💪 phase/solved Post is done 🧑 semver/major This is a change 🦋 type/enhancement This is great to have
Development

Successfully merging this pull request may close these issues.

2 participants