refactor(optimizer): nicer module finding + fix dev

- refactored manifest generation
- dev server collects QRL parents during ssr build
- optimizer also stores dev info for noop QRLs
- dev server also uses dev info as an extra fallback
- dev source URLs are kept similar looking to the source tree
- QRLs are annotated with parent so they can be built on demand

packages/docs/src/routes/api/qwik-optimizer/api.json

@@ -400,7 +400,7 @@
         }
       ],
       "kind": "Interface",
-      "content": "```typescript\nexport interface QwikManifest \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[bundles](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[fileName: string\\]: [QwikBundle](#qwikbundle)<!-- -->; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[injections?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n[GlobalInjections](#globalinjections)<!-- -->\\[\\]\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[manifestHash](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[mapping](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: string; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[options?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ target?: string; buildMode?: string; entryStrategy?: { \\[key: string\\]: any; }; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[platform?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[name: string\\]: string; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[symbols](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: [QwikSymbol](#qwiksymbol)<!-- -->; }\n\n\n</td><td>\n\n\n</td></tr>\n<tr><td>\n\n[version](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>",
+      "content": "The metadata of the build. One of its uses is storing where QRL symbols are located.\n\n\n```typescript\nexport interface QwikManifest \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[bundles](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[fileName: string\\]: [QwikBundle](#qwikbundle)<!-- -->; }\n\n\n</td><td>\n\nAll code bundles, used to know the import graph\n\n\n</td></tr>\n<tr><td>\n\n[injections?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n[GlobalInjections](#globalinjections)<!-- -->\\[\\]\n\n\n</td><td>\n\n_(Optional)_ CSS etc to inject in the document head\n\n\n</td></tr>\n<tr><td>\n\n[manifestHash](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\nContent hash of the manifest, if this changes, the code changed\n\n\n</td></tr>\n<tr><td>\n\n[mapping](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: string; }\n\n\n</td><td>\n\nWhere QRLs are located\n\n\n</td></tr>\n<tr><td>\n\n[options?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ target?: string; buildMode?: string; entryStrategy?: { \\[key: string\\]: any; }; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[platform?](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[name: string\\]: string; }\n\n\n</td><td>\n\n_(Optional)_\n\n\n</td></tr>\n<tr><td>\n\n[symbols](#)\n\n\n</td><td>\n\n\n</td><td>\n\n{ \\[symbolName: string\\]: [QwikSymbol](#qwiksymbol)<!-- -->; }\n\n\n</td><td>\n\nQRL symbols\n\n\n</td></tr>\n<tr><td>\n\n[version](#)\n\n\n</td><td>\n\n\n</td><td>\n\nstring\n\n\n</td><td>\n\n\n</td></tr>\n</tbody></table>",
       "editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/optimizer/src/types.ts",
       "mdFile": "qwik.qwikmanifest.md"
     },
@@ -644,7 +644,7 @@
         }
       ],
       "kind": "TypeAlias",
-      "content": "```typescript\nexport type SymbolMapperFn = (symbolName: string, mapper: SymbolMapper | undefined) => readonly [symbol: string, chunk: string] | undefined;\n```\n**References:** [SymbolMapper](#symbolmapper)",
+      "content": "```typescript\nexport type SymbolMapperFn = (symbolName: string, mapper: SymbolMapper | undefined, parent?: string) => readonly [symbol: string, chunk: string] | undefined;\n```\n**References:** [SymbolMapper](#symbolmapper)",
       "editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/optimizer/src/types.ts",
       "mdFile": "qwik.symbolmapperfn.md"
     },

packages/docs/src/routes/api/qwik-optimizer/index.md

@@ -1547,6 +1547,8 @@ _(Optional)_
 
 ## QwikManifest
 
+The metadata of the build. One of its uses is storing where QRL symbols are located.
+
 ```typescript
 export interface QwikManifest
 ```
@@ -1580,6 +1582,8 @@ Description
 
 </td><td>
 
+All code bundles, used to know the import graph
+
 </td></tr>
 <tr><td>
 
@@ -1593,7 +1597,7 @@ Description
 
 </td><td>
 
-_(Optional)_
+_(Optional)_ CSS etc to inject in the document head
 
 </td></tr>
 <tr><td>
@@ -1608,6 +1612,8 @@ string
 
 </td><td>
 
+Content hash of the manifest, if this changes, the code changed
+
 </td></tr>
 <tr><td>
 
@@ -1621,6 +1627,8 @@ string
 
 </td><td>
 
+Where QRLs are located
+
 </td></tr>
 <tr><td>
 
@@ -1664,6 +1672,8 @@ _(Optional)_
 
 </td><td>
 
+QRL symbols
+
 </td></tr>
 <tr><td>
 
@@ -2758,6 +2768,7 @@ export type SymbolMapper = Record<
 export type SymbolMapperFn = (
   symbolName: string,
   mapper: SymbolMapper | undefined,
+  parent?: string,
 ) => readonly [symbol: string, chunk: string] | undefined;
 ```
 

packages/docs/src/routes/api/qwik/api.json

@@ -544,7 +544,7 @@
         }
       ],
       "kind": "Interface",
-      "content": "Low-level API for platform abstraction.\n\nDifferent platforms (browser, node, service workers) may have different ways of handling things such as `requestAnimationFrame` and imports. To make Qwik platform-independent Qwik uses the `CorePlatform` API to access the platform API.\n\n`CorePlatform` also is responsible for importing symbols. The import map is different on the client (browser) then on the server. For this reason, the server has a manifest that is used to map symbols to javascript chunks. The manifest is encapsulated in `CorePlatform`<!-- -->, for this reason, the `CorePlatform` can't be global as there may be multiple applications running at server concurrently.\n\nThis is a low-level API and there should not be a need for you to access this.\n\n\n```typescript\nexport interface CorePlatform \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[chunkForSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(symbolName: string, chunk: string \\| null) =&gt; readonly \\[symbol: string, chunk: string\\] \\| undefined\n\n\n</td><td>\n\nRetrieve chunk name for the symbol.\n\nWhen the application is running on the server the symbols may be imported from different files (as server build is typically a single javascript chunk.) For this reason, it is necessary to convert the chunks from server format to client (browser) format. This is done by looking up symbols (which are globally unique) in the manifest. (Manifest is the mapping of symbols to the client chunk names.)\n\n\n</td></tr>\n<tr><td>\n\n[importSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(containerEl: Element \\| undefined, url: string \\| URL \\| undefined \\| null, symbol: string) =&gt; [ValueOrPromise](#valueorpromise)<!-- -->&lt;any&gt;\n\n\n</td><td>\n\nRetrieve a symbol value from QRL.\n\nQwik needs to lazy load data and closures. For this Qwik uses QRLs that are serializable references of resources that are needed. The QRLs contain all the information necessary to retrieve the reference using `importSymbol`<!-- -->.\n\nWhy not use `import()`<!-- -->? Because `import()` is relative to the current file, and the current file is always the Qwik framework. So QRLs have additional information that allows them to serialize imports relative to application base rather than the Qwik framework file.\n\n\n</td></tr>\n<tr><td>\n\n[isServer](#)\n\n\n</td><td>\n\n\n</td><td>\n\nboolean\n\n\n</td><td>\n\nTrue of running on the server platform.\n\n\n</td></tr>\n<tr><td>\n\n[nextTick](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next tick.\n\n\n</td></tr>\n<tr><td>\n\n[raf](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next request-animation-frame.\n\n\n</td></tr>\n</tbody></table>",
+      "content": "Low-level API for platform abstraction.\n\nDifferent platforms (browser, node, service workers) may have different ways of handling things such as `requestAnimationFrame` and imports. To make Qwik platform-independent Qwik uses the `CorePlatform` API to access the platform API.\n\n`CorePlatform` also is responsible for importing symbols. The import map is different on the client (browser) then on the server. For this reason, the server has a manifest that is used to map symbols to javascript chunks. The manifest is encapsulated in `CorePlatform`<!-- -->, for this reason, the `CorePlatform` can't be global as there may be multiple applications running at server concurrently.\n\nThis is a low-level API and there should not be a need for you to access this.\n\n\n```typescript\nexport interface CorePlatform \n```\n\n\n<table><thead><tr><th>\n\nProperty\n\n\n</th><th>\n\nModifiers\n\n\n</th><th>\n\nType\n\n\n</th><th>\n\nDescription\n\n\n</th></tr></thead>\n<tbody><tr><td>\n\n[chunkForSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(symbolName: string, chunk: string \\| null, parent?: string) =&gt; readonly \\[symbol: string, chunk: string\\] \\| undefined\n\n\n</td><td>\n\nRetrieve chunk name for the symbol.\n\nWhen the application is running on the server the symbols may be imported from different files (as server build is typically a single javascript chunk.) For this reason, it is necessary to convert the chunks from server format to client (browser) format. This is done by looking up symbols (which are globally unique) in the manifest. (Manifest is the mapping of symbols to the client chunk names.)\n\n\n</td></tr>\n<tr><td>\n\n[importSymbol](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(containerEl: Element \\| undefined, url: string \\| URL \\| undefined \\| null, symbol: string) =&gt; [ValueOrPromise](#valueorpromise)<!-- -->&lt;any&gt;\n\n\n</td><td>\n\nRetrieve a symbol value from QRL.\n\nQwik needs to lazy load data and closures. For this Qwik uses QRLs that are serializable references of resources that are needed. The QRLs contain all the information necessary to retrieve the reference using `importSymbol`<!-- -->.\n\nWhy not use `import()`<!-- -->? Because `import()` is relative to the current file, and the current file is always the Qwik framework. So QRLs have additional information that allows them to serialize imports relative to application base rather than the Qwik framework file.\n\n\n</td></tr>\n<tr><td>\n\n[isServer](#)\n\n\n</td><td>\n\n\n</td><td>\n\nboolean\n\n\n</td><td>\n\nTrue of running on the server platform.\n\n\n</td></tr>\n<tr><td>\n\n[nextTick](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next tick.\n\n\n</td></tr>\n<tr><td>\n\n[raf](#)\n\n\n</td><td>\n\n\n</td><td>\n\n(fn: () =&gt; any) =&gt; Promise&lt;any&gt;\n\n\n</td><td>\n\nPerform operation on next request-animation-frame.\n\n\n</td></tr>\n</tbody></table>",
       "editUrl": "https://github.com/QwikDev/qwik/tree/main/packages/qwik/src/core/platform/types.ts",
       "mdFile": "qwik.coreplatform.md"
     },

packages/docs/src/routes/api/qwik/index.md

@@ -1558,7 +1558,7 @@ Description
 
 </td><td>
 
-(symbolName: string, chunk: string \| null) =&gt; readonly [symbol: string, chunk: string] \| undefined
+(symbolName: string, chunk: string \| null, parent?: string) =&gt; readonly [symbol: string, chunk: string] \| undefined
 
 </td><td>
 

packages/qwik-city/runtime/src/spa-shim.ts

@@ -42,7 +42,7 @@ const shim = async (path: string, symbol: string) => {
       const imp = new Function('url', 'return import(url)');
       (await imp(url.href))[symbol](currentScript);
     } else {
-      (await import(url.href))[symbol](currentScript);
+      (await import(/* @vite-ignore */ url.href))[symbol](currentScript);
     }
   }
 };

packages/qwik/src/core/api.md

@@ -142,7 +142,7 @@ export interface ContextId<STATE> {
 
 // @public
 export interface CorePlatform {
-    chunkForSymbol: (symbolName: string, chunk: string | null) => readonly [symbol: string, chunk: string] | undefined;
+    chunkForSymbol: (symbolName: string, chunk: string | null, parent?: string) => readonly [symbol: string, chunk: string] | undefined;
     importSymbol: (containerEl: Element | undefined, url: string | URL | undefined | null, symbol: string) => ValueOrPromise<any>;
     isServer: boolean;
     nextTick: (fn: () => any) => Promise<any>;
@@ -542,6 +542,9 @@ export type NativeWheelEvent = WheelEvent;
 // @internal (undocumented)
 export const _noopQrl: <T>(symbolName: string, lexicalScopeCapture?: any[]) => QRL<T>;
 
+// @internal (undocumented)
+export const _noopQrlDEV: <T>(symbolName: string, opts: QRLDev, lexicalScopeCapture?: any[]) => QRL<T>;
+
 // @public
 export type NoSerialize<T> = (T & {
     __no_serialize__: true;

packages/qwik/src/core/internal.ts

@@ -1,5 +1,5 @@
 export { _pauseFromContexts, _serializeData } from './container/pause';
-export { _noopQrl, _regSymbol } from './qrl/qrl';
+export { _noopQrl, _noopQrlDEV, _regSymbol } from './qrl/qrl';
 export { _renderSSR } from './render/ssr/render-ssr';
 export { _hW } from './render/dom/notify-render';
 export { _wrapSignal, _wrapProp } from './state/signal';

packages/qwik/src/core/platform/types.ts

@@ -97,7 +97,8 @@ export interface CorePlatform {
   // </docs>
   chunkForSymbol: (
     symbolName: string,
-    chunk: string | null
+    chunk: string | null,
+    parent?: string
   ) => readonly [symbol: string, chunk: string] | undefined;
 }
 

packages/qwik/src/core/qrl/qrl.ts

@@ -124,6 +124,17 @@ export const _noopQrl = <T>(
   return createQRL<T>(null, symbolName, null, null, null, lexicalScopeCapture, null);
 };
 
+/** @internal */
+export const _noopQrlDEV = <T>(
+  symbolName: string,
+  opts: QRLDev,
+  lexicalScopeCapture: any[] = EMPTY_ARRAY
+): QRL<T> => {
+  const newQrl = _noopQrl(symbolName, lexicalScopeCapture) as QRLInternal<T>;
+  newQrl.dev = opts;
+  return newQrl;
+};
+
 /** @internal */
 export const qrlDEV = <T = any>(
   chunkOrFn: string | (() => Promise<any>),
@@ -163,12 +174,14 @@ export const serializeQRL = (qrl: QRLInternal, opts: QRLSerializeOptions = {}) =
   const platform = getPlatform();
 
   if (platform) {
-    const result = platform.chunkForSymbol(refSymbol, chunk);
+    const result = platform.chunkForSymbol(refSymbol, chunk, qrl.dev?.file);
     if (result) {
       chunk = result[1];
       if (!qrl.$refSymbol$) {
         symbol = result[0];
       }
+    } else {
+      console.error('serializeQRL: Cannot resolve symbol', symbol, 'in', chunk, qrl.dev?.file);
     }
   }
 
@@ -197,7 +210,7 @@ export const serializeQRL = (qrl: QRLInternal, opts: QRLSerializeOptions = {}) =
       throwErrorAndStop('Sync QRL without containerState');
     }
   }
-  let output = `${chunk}#${symbol}`;
+  let output = `${encodeURI(chunk)}#${symbol}`;
   const capture = qrl.$capture$;
   const captureRef = qrl.$captureRef$;
   if (captureRef && captureRef.length) {

packages/qwik/src/core/qrl/qrl.unit.ts

@@ -99,6 +99,12 @@ describe('serialization', () => {
       serializeQRL(createQRL('c', 's1', null, null, [1 as any, '2'], null, null)),
       'c#s1[1 2]'
     );
+    assert.equal(
+      serializeQRL(
+        createQRL('src/routes/[...index]/c', 's1', null, null, [1 as any, '2'], null, null)
+      ),
+      'src/routes/%5B...index%5D/c#s1[1 2]'
+    );
   });
 
   test('should parse reference', () => {

packages/qwik/src/core/use/use-task.ts

@@ -793,6 +793,10 @@ const getTaskHandlerQrl = (task: SubscriberEffect): QRL<(ev: Event) => void> =>
     [task],
     taskQrl.$symbol$
   );
+  // Needed for chunk lookup in dev mode
+  if (taskQrl.dev) {
+    taskHandler.dev = taskQrl.dev;
+  }
   return taskHandler;
 };
 

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_derived_signals_children.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2728
+assertion_line: 2749
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_derived_signals_cmp.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2861
+assertion_line: 2882
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_derived_signals_complext_children.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2830
+assertion_line: 2850
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_derived_signals_div.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2640
+assertion_line: 2661
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_derived_signals_multiple_children.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2783
+assertion_line: 2804
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_dev_mode_inlined.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2187
+assertion_line: 2208
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_immutable_function_components.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2469
+assertion_line: 2490
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_issue_33443.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2914
+assertion_line: 2935
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_issue_4438.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2704
+assertion_line: 2725
 expression: output
 ---
 ==INPUT==

packages/qwik/src/optimizer/core/src/snapshots/qwik_core__test__example_mutable_children.snap

@@ -1,6 +1,6 @@
 ---
 source: packages/qwik/src/optimizer/core/src/test.rs
-assertion_line: 2361
+assertion_line: 2382
 expression: output
 ---
 ==INPUT==