Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support require.context #4399

Merged
merged 10 commits into from
Apr 18, 2023
Merged

support require.context #4399

merged 10 commits into from
Apr 18, 2023

Conversation

ForsakenHarmony
Copy link
Contributor

@ForsakenHarmony ForsakenHarmony commented Mar 30, 2023

fix WEB-535

@vercel
Copy link

vercel bot commented Mar 30, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
turbo-site ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 18, 2023 3:29pm
10 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-cra-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-gatsby-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-native-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-nonmonorepo ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-svelte-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-tailwind-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm
examples-vite-web ⬜️ Ignored (Inspect) Apr 18, 2023 3:29pm

@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2023

✅ This changes can build next-swc

@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2023

🟢 CI successful 🟢

Thanks

@ForsakenHarmony ForsakenHarmony force-pushed the hrmny/web-535-require-context branch from dd8a7f3 to ebc54b7 Compare April 4, 2023 15:47
@ForsakenHarmony ForsakenHarmony force-pushed the hrmny/web-535-require-context branch from ebc54b7 to 337bad1 Compare April 11, 2023 17:59
@ForsakenHarmony ForsakenHarmony force-pushed the hrmny/web-535-require-context branch from 337bad1 to 75b3783 Compare April 11, 2023 18:26
@ForsakenHarmony ForsakenHarmony changed the title support require.context (WIP) support require.context Apr 11, 2023
@ForsakenHarmony ForsakenHarmony marked this pull request as ready for review April 11, 2023 20:37
@ForsakenHarmony ForsakenHarmony requested a review from a team as a code owner April 11, 2023 20:37
Copy link
Member

@sokra sokra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach is a bit inefficient from incremental compilation point of view, but we can move that to tech dept:

  • adding/removing a file in the root directory with walk the whole directory again (list_dir is recomputed)
    • This could be solved by using a tree reprensentation instead of FlatDirList
  • The module using require.context need to be re-analyzed once a file is added/removed from the context
    • This could be solved by moving the context map into a separate module
    • And computing the require.context for static analysis only when a key is accessed from it. And making this a dedicated tt function

crates/turbopack-dev/js/src/runtime.js Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
DirectoryEntry::File(path) => {
if let Some(relative_path) = root.get_relative_path_to(&*path.await?) {
if filter.is_match(&relative_path) {
list.push((relative_path, *path));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It need to apply some resolve options here. e. g. extensions.

e.g. a file ./abc/index.ts should have multiple keys: ./abc/index.ts ./abc/index ./abc/ or ./abc

a package.json with main or exports would also influence that. (webpack doesn't handle that case)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would the package.json influence this?
The API seems to be file path based, so webpack's behavior kinda makes sense to me

crates/turbopack-ecmascript/src/references/mod.rs Outdated Show resolved Hide resolved
_,
box JsValue::WellKnownFunction(WellKnownFunctionKind::RequireContext),
args,
) => require_context_visitor(origin, args, in_try).await?,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static evaluation for require.context... You are really (over)killing it...

webpack doesn't even support that.

So I can do something fancy like:

const context = require.context("...");
const resolved = context.resolve("./a");
const imported = require(resolved); // <- usually this would be too dynamic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, the static evaluation was the reason I didn't put the context into a separate asset/module

crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
crates/turbopack-ecmascript/src/analyzer/mod.rs Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Benchmark for 8966c65

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8745.17µs ± 49.23µs 8740.46µs ± 46.95µs -0.05%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7793.91µs ± 40.27µs 8125.34µs ± 287.04µs +4.25%
bench_startup/Turbopack CSR/1000 modules 854.38ms ± 0.91ms 855.29ms ± 2.39ms +0.11%

@github-actions
Copy link
Contributor

Benchmark for 3aada5b

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 9423.77µs ± 30.42µs 9329.25µs ± 45.17µs -1.00%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8383.25µs ± 84.63µs 9335.56µs ± 1100.88µs +11.36%
bench_startup/Turbopack CSR/1000 modules 892.95ms ± 3.93ms 898.91ms ± 9.19ms +0.67%

@github-actions
Copy link
Contributor

Benchmark for 21e66c2

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8964.89µs ± 62.41µs 8887.06µs ± 35.55µs -0.87%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8094.30µs ± 31.97µs 8623.23µs ± 713.97µs +6.53%
bench_startup/Turbopack CSR/1000 modules 870.40ms ± 1.27ms 863.45ms ± 6.81ms -0.80%

@github-actions
Copy link
Contributor

Benchmark for 4b488f6

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 9415.80µs ± 31.19µs 9453.59µs ± 27.29µs +0.40%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8426.37µs ± 31.81µs 8575.28µs ± 59.34µs +1.77%
bench_startup/Turbopack CSR/1000 modules 888.79ms ± 1.97ms 879.53ms ± 3.07ms -1.04%

@ForsakenHarmony ForsakenHarmony force-pushed the hrmny/web-535-require-context branch from 9682829 to ead9225 Compare April 18, 2023 15:24
@ForsakenHarmony ForsakenHarmony dismissed alexkirsz’s stale review April 18, 2023 15:55

comments have been addressed

@ForsakenHarmony ForsakenHarmony merged commit ffe82d4 into main Apr 18, 2023
@ForsakenHarmony ForsakenHarmony deleted the hrmny/web-535-require-context branch April 18, 2023 15:56
sokra added a commit to vercel/next.js that referenced this pull request Apr 20, 2023
### What?

add support for blur placeholder generation to turbopack

add `StructuredImageModuleType` which is used with `ModuleType::Custom`
to allow importing an image as `{ url, width, height, blurDataURL,
blurWidth, blurHeight }`

in contrast to next.js with webpack this will also generate blur
placeholder in development instead of using a _next/image reference.
This should lead to more production-like experience (at the cost of a
little bit of compilation time).

turbo PR: vercel/turborepo#4621

### Why?

Turbopack was crashing on `placeholder="blur"` before.

fixes WEB-534

### Turbopack changes

* vercel/turborepo#4521 <!-- OJ Kwon -
feat(contextcondition): support InPath contextcondition -->
* vercel/turborepo#4601 <!-- Alex Kirszenberg -
Chunking Context Refactor pt. 3: Address PR comments from pt. 2 -->
* vercel/turborepo#4623 <!-- Tobias Koppers -
exclude turborepo from turbopack bench tests -->
* vercel/turborepo#4399 <!-- Leah - support
require.context -->
* vercel/turborepo#4610 <!-- OJ Kwon - test(subset):
add mdx test into subset -->
* vercel/turborepo#4624 <!-- Tobias Koppers - run
benchmarks on windows and macOS too -->
* vercel/turborepo#4620 <!-- Alex Kirszenberg - Make
ContainmentTree fully generic -->
* vercel/turborepo#4600 <!-- Tobias Koppers - add
getChunkPath method -->
* vercel/turborepo#4621 <!-- Tobias Koppers - add
turbopack-image -->
* vercel/turborepo#4639 <!-- Tobias Koppers -
restrict snapshot path for windows path length limit -->
* vercel/turborepo#4641 <!-- Tobias Koppers - put
webp behind a feature flag -->
NicholasLYang pushed a commit to NicholasLYang/turbo that referenced this pull request Apr 21, 2023
ForsakenHarmony added a commit to vercel/next.js that referenced this pull request Jul 25, 2024
ForsakenHarmony added a commit to vercel/next.js that referenced this pull request Jul 29, 2024
ForsakenHarmony added a commit to vercel/next.js that referenced this pull request Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants