Skip to content

Commit

Permalink
refactor(optimizer): nicer module finding + fix dev
Browse files Browse the repository at this point in the history
- refactored manifest generation
- dev server collects QRL parents during ssr build
- optimizer also stores dev info for noop QRLs
- dev server also uses dev info as an extra fallback
- dev source URLs are kept similar looking to the source tree
- QRLs are annotated with parent so they can be built on demand
  • Loading branch information
wmertens committed Jun 17, 2024
1 parent c53e8a5 commit bf75494
Show file tree
Hide file tree
Showing 40 changed files with 457 additions and 255 deletions.
4 changes: 2 additions & 2 deletions packages/docs/src/routes/api/qwik-optimizer/api.json
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,7 @@
}
],
"kind": "Interface",
"content": "```typescript\nexport interface QwikManifest \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[bundles](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[fileName: string\\]: [QwikBundle](#qwikbundle)<!-- -->; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[injections?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n[GlobalInjections](#globalinjections)<!-- -->\\[\\]\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[manifestHash](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[mapping](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: string; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[options?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ target?: string; buildMode?: string; entryStrategy?: { \\[key: string\\]: any; }; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[platform?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[name: string\\]: string; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[symbols](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: [QwikSymbol](#qwiksymbol)<!-- -->; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[version](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>",
"content": "The metadata of the build. One of its uses is storing where QRL symbols are located.\n\n\n```typescript\nexport interface QwikManifest \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[bundles](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[fileName: string\\]: [QwikBundle](#qwikbundle)<!-- -->; }\n\n\n</td><td>\n\nAll code bundles, used to know the import graph\n\n\n</td></tr>\n<tr><td>\n\n[injections?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n[GlobalInjections](#globalinjections)<!-- -->\\[\\]\n\n\n</td><td>\n\n_(Optional)_ CSS etc to inject in the document head\n\n\n</td></tr>\n<tr><td>\n\n[manifestHash](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\nContent hash of the manifest, if this changes, the code changed\n\n\n</td></tr>\n<tr><td>\n\n[mapping](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: string; }\n\n\n</td><td>\n\nWhere QRLs are located\n\n\n</td></tr>\n<tr><td>\n\n[options?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ target?: string; buildMode?: string; entryStrategy?: { \\[key: string\\]: any; }; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[platform?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[name: string\\]: string; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[symbols](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: [QwikSymbol](#qwiksymbol)<!-- -->; }\n\n\n</td><td>\n\nQRL symbols\n\n\n</td></tr>\n<tr><td>\n\n[version](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>",
"editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/optimizer/src/types.ts",
"mdFile": "qwik.qwikmanifest.md"
},
Expand Down Expand Up @@ -644,7 +644,7 @@
}
],
"kind": "TypeAlias",
"content": "```typescript\nexport type SymbolMapperFn = (symbolName: string, mapper: SymbolMapper | undefined) => readonly [symbol: string, chunk: string] | undefined;\n```\n**References:** [SymbolMapper](#symbolmapper)",
"content": "```typescript\nexport type SymbolMapperFn = (symbolName: string, mapper: SymbolMapper | undefined, parent?: string) => readonly [symbol: string, chunk: string] | undefined;\n```\n**References:** [SymbolMapper](#symbolmapper)",
"editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/optimizer/src/types.ts",
"mdFile": "qwik.symbolmapperfn.md"
},
Expand Down
13 changes: 12 additions & 1 deletion packages/docs/src/routes/api/qwik-optimizer/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1547,6 +1547,8 @@ _(Optional)_

## QwikManifest

The metadata of the build. One of its uses is storing where QRL symbols are located.

```typescript
export interface QwikManifest
```
Expand Down Expand Up @@ -1580,6 +1582,8 @@ Description

</td><td>

All code bundles, used to know the import graph

</td></tr>
<tr><td>

Expand All @@ -1593,7 +1597,7 @@ Description

</td><td>

_(Optional)_
_(Optional)_ CSS etc to inject in the document head

</td></tr>
<tr><td>
Expand All @@ -1608,6 +1612,8 @@ string

</td><td>

Content hash of the manifest, if this changes, the code changed

</td></tr>
<tr><td>

Expand All @@ -1621,6 +1627,8 @@ string

</td><td>

Where QRLs are located

</td></tr>
<tr><td>

Expand Down Expand Up @@ -1664,6 +1672,8 @@ _(Optional)_

</td><td>

QRL symbols

</td></tr>
<tr><td>

Expand Down Expand Up @@ -2758,6 +2768,7 @@ export type SymbolMapper = Record<
export type SymbolMapperFn = (
symbolName: string,
mapper: SymbolMapper | undefined,
parent?: string,
) => readonly [symbol: string, chunk: string] | undefined;
```

Expand Down
2 changes: 1 addition & 1 deletion packages/docs/src/routes/api/qwik/api.json
Original file line number Diff line number Diff line change
Expand Up @@ -544,7 +544,7 @@
}
],
"kind": "Interface",
"content": "Low-level API for platform abstraction.\n\nDifferent platforms (browser, node, service workers) may have different ways of handling things such as `requestAnimationFrame` and imports. To make Qwik platform-independent Qwik uses the `CorePlatform` API to access the platform API.\n\n`CorePlatform` also is responsible for importing symbols. The import map is different on the client (browser) then on the server. For this reason, the server has a manifest that is used to map symbols to javascript chunks. The manifest is encapsulated in `CorePlatform`<!-- -->, for this reason, the `CorePlatform` can't be global as there may be multiple applications running at server concurrently.\n\nThis is a low-level API and there should not be a need for you to access this.\n\n\n```typescript\nexport interface CorePlatform \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[chunkForSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(symbolName: string, chunk: string \\| null) =&gt; readonly \\[symbol: string, chunk: string\\] \\| undefined\n\n\n</td><td>\n\nRetrieve chunk name for the symbol.\n\nWhen the application is running on the server the symbols may be imported from different files (as server build is typically a single javascript chunk.) For this reason, it is necessary to convert the chunks from server format to client (browser) format. This is done by looking up symbols (which are globally unique) in the manifest. (Manifest is the mapping of symbols to the client chunk names.)\n\n\n</td></tr>\n<tr><td>\n\n[importSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(containerEl: Element \\| undefined, url: string \\| URL \\| undefined \\| null, symbol: string) =&gt; [ValueOrPromise](#valueorpromise)<!-- -->&lt;any&gt;\n\n\n</td><td>\n\nRetrieve a symbol value from QRL.\n\nQwik needs to lazy load data and closures. For this Qwik uses QRLs that are serializable references of resources that are needed. The QRLs contain all the information necessary to retrieve the reference using `importSymbol`<!-- -->.\n\nWhy not use `import()`<!-- -->? Because `import()` is relative to the current file, and the current file is always the Qwik framework. So QRLs have additional information that allows them to serialize imports relative to application base rather than the Qwik framework file.\n\n\n</td></tr>\n<tr><td>\n\n[isServer](#)\n\n\n</td><td>\n\n\n</td><td>\n\nboolean\n\n\n</td><td>\n\nTrue of running on the server platform.\n\n\n</td></tr>\n<tr><td>\n\n[nextTick](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next tick.\n\n\n</td></tr>\n<tr><td>\n\n[raf](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next request-animation-frame.\n\n\n</td></tr>\n</tbody></table>",
"content": "Low-level API for platform abstraction.\n\nDifferent platforms (browser, node, service workers) may have different ways of handling things such as `requestAnimationFrame` and imports. To make Qwik platform-independent Qwik uses the `CorePlatform` API to access the platform API.\n\n`CorePlatform` also is responsible for importing symbols. The import map is different on the client (browser) then on the server. For this reason, the server has a manifest that is used to map symbols to javascript chunks. The manifest is encapsulated in `CorePlatform`<!-- -->, for this reason, the `CorePlatform` can't be global as there may be multiple applications running at server concurrently.\n\nThis is a low-level API and there should not be a need for you to access this.\n\n\n```typescript\nexport interface CorePlatform \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[chunkForSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(symbolName: string, chunk: string \\| null, parent?: string) =&gt; readonly \\[symbol: string, chunk: string\\] \\| undefined\n\n\n</td><td>\n\nRetrieve chunk name for the symbol.\n\nWhen the application is running on the server the symbols may be imported from different files (as server build is typically a single javascript chunk.) For this reason, it is necessary to convert the chunks from server format to client (browser) format. This is done by looking up symbols (which are globally unique) in the manifest. (Manifest is the mapping of symbols to the client chunk names.)\n\n\n</td></tr>\n<tr><td>\n\n[importSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(containerEl: Element \\| undefined, url: string \\| URL \\| undefined \\| null, symbol: string) =&gt; [ValueOrPromise](#valueorpromise)<!-- -->&lt;any&gt;\n\n\n</td><td>\n\nRetrieve a symbol value from QRL.\n\nQwik needs to lazy load data and closures. For this Qwik uses QRLs that are serializable references of resources that are needed. The QRLs contain all the information necessary to retrieve the reference using `importSymbol`<!-- -->.\n\nWhy not use `import()`<!-- -->? Because `import()` is relative to the current file, and the current file is always the Qwik framework. So QRLs have additional information that allows them to serialize imports relative to application base rather than the Qwik framework file.\n\n\n</td></tr>\n<tr><td>\n\n[isServer](#)\n\n\n</td><td>\n\n\n</td><td>\n\nboolean\n\n\n</td><td>\n\nTrue of running on the server platform.\n\n\n</td></tr>\n<tr><td>\n\n[nextTick](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next tick.\n\n\n</td></tr>\n<tr><td>\n\n[raf](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next request-animation-frame.\n\n\n</td></tr>\n</tbody></table>",
"editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/platform/types.ts",
"mdFile": "qwik.coreplatform.md"
},
Expand Down
2 changes: 1 addition & 1 deletion packages/docs/src/routes/api/qwik/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1558,7 +1558,7 @@ Description
</td><td>
(symbolName: string, chunk: string \| null) =&gt; readonly [symbol: string, chunk: string] \| undefined
(symbolName: string, chunk: string \| null, parent?: string) =&gt; readonly [symbol: string, chunk: string] \| undefined
</td><td>
Expand Down
2 changes: 1 addition & 1 deletion packages/qwik-city/runtime/src/spa-shim.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ const shim = async (path: string, symbol: string) => {
const imp = new Function('url', 'return import(url)');
(await imp(url.href))[symbol](currentScript);
} else {
(await import(url.href))[symbol](currentScript);
(await import(/* @vite-ignore */ url.href))[symbol](currentScript);
}
}
};
5 changes: 4 additions & 1 deletion packages/qwik/src/core/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ export interface ContextId<STATE> {

// @public
export interface CorePlatform {
chunkForSymbol: (symbolName: string, chunk: string | null) => readonly [symbol: string, chunk: string] | undefined;
chunkForSymbol: (symbolName: string, chunk: string | null, parent?: string) => readonly [symbol: string, chunk: string] | undefined;
importSymbol: (containerEl: Element | undefined, url: string | URL | undefined | null, symbol: string) => ValueOrPromise<any>;
isServer: boolean;
nextTick: (fn: () => any) => Promise<any>;
Expand Down Expand Up @@ -542,6 +542,9 @@ export type NativeWheelEvent = WheelEvent;
// @internal (undocumented)
export const _noopQrl: <T>(symbolName: string, lexicalScopeCapture?: any[]) => QRL<T>;

// @internal (undocumented)
export const _noopQrlDEV: <T>(symbolName: string, opts: QRLDev, lexicalScopeCapture?: any[]) => QRL<T>;

// @public
export type NoSerialize<T> = (T & {
__no_serialize__: true;
Expand Down
2 changes: 1 addition & 1 deletion packages/qwik/src/core/internal.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
export { _pauseFromContexts, _serializeData } from './container/pause';
export { _noopQrl, _regSymbol } from './qrl/qrl';
export { _noopQrl, _noopQrlDEV, _regSymbol } from './qrl/qrl';
export { _renderSSR } from './render/ssr/render-ssr';
export { _hW } from './render/dom/notify-render';
export { _wrapSignal, _wrapProp } from './state/signal';
Expand Down
3 changes: 2 additions & 1 deletion packages/qwik/src/core/platform/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@ export interface CorePlatform {
// </docs>
chunkForSymbol: (
symbolName: string,
chunk: string | null
chunk: string | null,
parent?: string
) => readonly [symbol: string, chunk: string] | undefined;
}

Expand Down
17 changes: 15 additions & 2 deletions packages/qwik/src/core/qrl/qrl.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,17 @@ export const _noopQrl = <T>(
return createQRL<T>(null, symbolName, null, null, null, lexicalScopeCapture, null);
};

/** @internal */
export const _noopQrlDEV = <T>(
symbolName: string,
opts: QRLDev,
lexicalScopeCapture: any[] = EMPTY_ARRAY
): QRL<T> => {
const newQrl = _noopQrl(symbolName, lexicalScopeCapture) as QRLInternal<T>;
newQrl.dev = opts;
return newQrl;
};

/** @internal */
export const qrlDEV = <T = any>(
chunkOrFn: string | (() => Promise<any>),
Expand Down Expand Up @@ -163,12 +174,14 @@ export const serializeQRL = (qrl: QRLInternal, opts: QRLSerializeOptions = {}) =
const platform = getPlatform();

if (platform) {
const result = platform.chunkForSymbol(refSymbol, chunk);
const result = platform.chunkForSymbol(refSymbol, chunk, qrl.dev?.file);
if (result) {
chunk = result[1];
if (!qrl.$refSymbol$) {
symbol = result[0];
}
} else {
console.error('serializeQRL: Cannot resolve symbol', symbol, 'in', chunk, qrl.dev?.file);
}
}

Expand Down Expand Up @@ -197,7 +210,7 @@ export const serializeQRL = (qrl: QRLInternal, opts: QRLSerializeOptions = {}) =
throwErrorAndStop('Sync QRL without containerState');
}
}
let output = `${chunk}#${symbol}`;
let output = `${encodeURI(chunk)}#${symbol}`;
const capture = qrl.$capture$;
const captureRef = qrl.$captureRef$;
if (captureRef && captureRef.length) {
Expand Down
6 changes: 6 additions & 0 deletions packages/qwik/src/core/qrl/qrl.unit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,12 @@ describe('serialization', () => {
serializeQRL(createQRL('c', 's1', null, null, [1 as any, '2'], null, null)),
'c#s1[1 2]'
);
assert.equal(
serializeQRL(
createQRL('src/routes/[...index]/c', 's1', null, null, [1 as any, '2'], null, null)
),
'src/routes/%5B...index%5D/c#s1[1 2]'
);
});

test('should parse reference', () => {
Expand Down
4 changes: 4 additions & 0 deletions packages/qwik/src/core/use/use-task.ts
Original file line number Diff line number Diff line change
Expand Up @@ -793,6 +793,10 @@ const getTaskHandlerQrl = (task: SubscriberEffect): QRL<(ev: Event) => void> =>
[task],
taskQrl.$symbol$
);
// Needed for chunk lookup in dev mode
if (taskQrl.dev) {
taskHandler.dev = taskQrl.dev;
}
return taskHandler;
};

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2728
assertion_line: 2749
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2861
assertion_line: 2882
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2830
assertion_line: 2850
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2640
assertion_line: 2661
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2783
assertion_line: 2804
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2187
assertion_line: 2208
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2469
assertion_line: 2490
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2914
assertion_line: 2935
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2704
assertion_line: 2725
expression: output
---
==INPUT==
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: packages/qwik/src/optimizer/core/src/test.rs
assertion_line: 2361
assertion_line: 2382
expression: output
---
==INPUT==
Expand Down
Loading

0 comments on commit bf75494

Please sign in to comment.