Server assets cannot be read on the server after build

[oxneat]

Describe the bug

Sveletekit PdfMake font import works fine locally but crushes when deployed to vercel

Reproduction

this is the code:

const fonts: TFontDictionary = {
    Futura: {
        //         FuturaPTBook.otf
        // FuturaPTCondBold.otf
        normal: `./static/fonts/FuturaPTBook.otf`,
        bold: `./static/fonts/FuturaPTBold.otf`
    }
};
const printer = new PdfPrinter(fonts);

i also created a fonts folder a the same place where the file exists and i imported the fonts using
import normal from "./fonts/FuturaPTBook.otf"; and i did this:

const fonts: TFontDictionary = {
    Futura: {
        //         FuturaPTBook.otf
        // FuturaPTCondBold.otf
        normal: path.join(".",normal),
        bold: path.join(".",bold)
    }
};

but it worked locally and it crashed on vercel

Logs

Error: ENOENT: no such file or directory, open '_app/immutable/assets/FuturaPTCondBold.35d3f580.otf'
    at Object.openSync (node:fs:601:3)
    at Object.readFileSync (node:fs:469:35)
    at PDFFontFactory.open (/var/task/vercel/path0/node_modules/@foliojs-fork/pdfkit/js/pdfkit.js:3384:16)
    at PDFDocument.font (/var/task/vercel/path0/node_modules/@foliojs-fork/pdfkit/js/pdfkit.js:3454:33)
    at FontProvider.provideFont (/var/task/vercel/path0/node_modules/pdfmake/src/fontProvider.js:62:58)
    at /var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:326:27
    at Array.forEach (<anonymous>)
    at measure (/var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:298:13)
    at TextTools.buildInlines (/var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:32:17)
    at DocMeasure.measureLeaf (/var/task/vercel/path0/node_modules/pdfmake/src/docMeasure.js:235:28) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '_app/immutable/assets/FuturaPTCondBold.35d3f580.otf'
}

System Info

System:
    OS: Linux 6.2 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (4) x64 Intel(R) Core(TM) i5-3360M CPU @ 2.80GHz
    Memory: 2.68 GB / 7.68 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 18.16.0 - ~/.nvm/versions/node/v18.16.0/bin/node
    Yarn: 3.6.0 - ~/.nvm/versions/node/v18.16.0/bin/yarn
    npm: 9.5.1 - ~/.nvm/versions/node/v18.16.0/bin/npm
    pnpm: 8.6.7 - ~/.nvm/versions/node/v18.16.0/bin/pnpm
  Browsers:
    Chrome: 115.0.5790.170
  npmPackages:
    @sveltejs/adapter-auto: ^2.0.0 => 2.1.0 
    @sveltejs/kit: ^1.20.4 => 1.22.3 
    svelte: ^4.0.5 => 4.0.5 
    vite: ^4.4.2 => 4.4.4

Severity

serious, but I can work around it

Additional Information

No response

[eltigerchino]

Please provide a minimal reproduction in the form of a repository that can be deployed to Vercel to reproduce this issue.

[oxneat]

@s3812497 Hello I am sorry for the delay i think we different times zones, here is the repo

https://github.com/oxneat/sVELTEKITvercelFonts

you can clone it. it reproduces the same error.

[eltigerchino]

This is an expected error because ./static/fonts/FuturaPTBook.otf is not a valid path after building. The contents of the /static folder get copied to the root of the build output. https://vitejs.dev/guide/assets.html#the-public-directory

Instead, you should store your assets in /lib and import them so that Vite processes them. This will resolve to the correct path. https://vitejs.dev/guide/assets.html#static-asset-handling

import futuraNormal from '$lib/fonts/FuturaPTBook.otf';
import futuraBold from '$lib/fonts/FuturaPTCondBold.otf';

const fonts: TFontDictionary = {
    Futura: {
        normal: futuraNormal,
        bold: futuraBold
    }
};

https://stackblitz.com/edit/github-xfpkrg?file=src%2Flib%2Fserver%2Fpdf%2Findex.ts

[oxneat]

@s3812497 Hello Tee Thanks for your help, i tried this and it worked locally but unfortunately it failed on vercel again and this is the error:
Error: ENOENT: no such file or directory, open './_app/immutable/assets/FuturaPTCondBold.35d3f580.otf' at Object.openSync (node:fs:601:3) at Object.readFileSync (node:fs:469:35) at PDFFontFactory.open (/var/task/vercel/path0/node_modules/@foliojs-fork/pdfkit/js/pdfkit.js:3384:16) at PDFDocument.font (/var/task/vercel/path0/node_modules/@foliojs-fork/pdfkit/js/pdfkit.js:3454:33) at FontProvider.provideFont (/var/task/vercel/path0/node_modules/pdfmake/src/fontProvider.js:62:58) at /var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:326:27 at Array.forEach (<anonymous>) at measure (/var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:298:13) at TextTools.buildInlines (/var/task/vercel/path0/node_modules/pdfmake/src/textTools.js:32:17) at DocMeasure.measureLeaf (/var/task/vercel/path0/node_modules/pdfmake/src/docMeasure.js:235:28) { errno: -2, syscall: 'open', code: 'ENOENT', path: './_app/immutable/assets/FuturaPTCondBold.35d3f580.otf' }

[eltigerchino]

I don't think we're copying the server assets over / resolving the path correctly for some adapters (node, vercel, etc.). So, trying to read them on the server causes the file to not be found.

[oxneat]

@s3812497 Additionally __dirname and __filename don't work

[eltigerchino]

related #4671

[diramazioni]

I'm a new user but... have you checked the files.assets in svelte.config.js? like

		files: {
			assets:  './src/lib/assets' 
		},

[ghostdevv]

Additionally __dirname and __filename don't work

I believe the output is esm so you would need import.meta.url

[oxneat]

@diramazioni @s3812497 this didn't work

I'm a new user but... have you checked the files.assets in svelte.config.js? like

		files: {
			assets:  './src/lib/assets' 
		},

[oxneat]

Additionally __dirname and __filename don't work

I believe the output is esm so you would need import.meta.url

@ghostdevv @s3812497 this also didn't work

[oxneat]

@s3812497 hello guys is there anything new about this bug?

[eltigerchino]

@oxneat Sorry about the long silence. I was stuck on this issue for a while but have now found the root cause.

Currently, SvelteKit treats server assets no differently from client assets. Hence, we need to add to the builder methods available and modify existing adapters to correctly copy over the server assets to the correct path on the server.

In the case of Vercel, all assets currently live in the .vercel/output/static/ directory as public assets. However, those aren't directly accessible by server code. They need to live in the .vercel/output/functions/fn.func/ directory.

[oxneat]

@s3812497 thanks for the feedback

[eltigerchino]

It seems that this is actually a known issue with a warning in the docs.
https://kit.svelte.dev/docs/adapter-vercel#troubleshooting

#8441 (review)

[wesbos]

I'm also running into this. I have a wasm file that needs to be included in a +server.ts route. How do I ensure it's included and accessible at runtime?

[eltigerchino]

I'm also running into this. I have a wasm file that needs to be included in a +server.ts route. How do I ensure it's included and accessible at runtime?

See the last paragraph in #10594 (comment)

It should be accessible if you can manually copy the files in.

[wesbos]

@s3812497 How do I do that? Do I need to modify the sveltekit built?

[eltigerchino]

How do I do that? Do I need to modify the sveltekit built?

@wesbos That's right. A sort of post-build script.

# you can replace your package.json build script with the below
vite build && cp -r .svelte-kit/output/server/_app .vercel/output/functions/fn.func

import { join } from "path";
import { FFmpeg } from "@ffmpeg.wasm/main";
import core from "@ffmpeg.wasm/core-mt";
import wasmPathAb from "@ffmpeg.wasm/core-mt/dist/core.wasm?url";
import { env } from "$env/dynamic/private";
import { dev } from "$app/environment";

let wasmPath = join(process.cwd(), wasmPathAb);

// when running `vite preview`, files are served from .svelte-kit
// see https://kit.svelte.dev/docs/building-your-app#preview-your-app
if (!dev && !env.VERCEL) {
  wasmPath = join(process.cwd(), '.svelte-kit/output/server', wasmPathAb)
}

export async function GET() {
  const ffmpeg = await FFmpeg.create({
    core,
    coreOptions: {
      wasmPath: wasmPath,
    },
  });
  return new Response(wasmPath);
}

As a side note, we should be able to copy the server assets correctly when building for split functions (adapter-vercel and adapter-netlify) because we now retain the vite server manifest. Using the manifest, we can know which files belong to which routes.

[wesbos]

@s3812497 Thank you - is the fn.func dir supposed to be named something? Or literally "fn.func"? I tried the script but I still have the same issue. I can see the file in _app/immutable/assets/ dir

[eltigerchino]

@s3812497 is the fn.func dir supposed to be named something? Or literally "fn.func"?

I believe so, unless you have split functions configured and have multiple function folders.

[HenkBourgonje]

@oxneat have you been able to get this to work for your font-related problem on Vercel? I'm running into the same thing right now.

[oxneat]

@oxneat have you been able to get this to work for your font-related problem on Vercel? I'm running into the same thing right now.

No unfortunately

[HenkBourgonje]

@oxneat too bad! Do you by any chance have any more ideas for the original problem this thread is about @eltigerchino?

[eltigerchino]

@oxneat too bad! Do you by any chance have any more ideas for the original problem this thread is about @eltigerchino?

It's easy enough to update adapter-vercel to copy over all server assets into the functions folder.

However, I'm struggling to figure out how to easily determine which assets should go with which functions when the split config is enabled. This groups routes into different Vercel functions based on the exported config constant in +page.js.

The vite manifest maps the server assets to the file paths of the +layout.server and +page.server files in your project folder. So it'd be ideal to copy over only the server assets that will be used by the single server layout or multiple nested server layouts and the page server load. We should also take into account when a layout breaks out of the nested hierarchy. I'm not sure if this will create too much overhead for the adapter and slow it down.

It'd be easier to copy over the same server assets to every serverless function but that would make your build very large and so is one of the reasons this was not implemented earlier.

[wesbos]

Still banging my head against the wall with this. Ended up having to copy + paste the files into the vercel output after build:

#!zx
const LOCAL_BUILD_PATH = '.svelte-kit/output/server/';
const VERCEL_BUILD_BASE_PATH = '.vercel/output/functions';
const vercelFuncDirs = (await $`ls -1d ${VERCEL_BUILD_BASE_PATH}/*.func`).stdout
	.split('\n')
	.map((v) => v.trim())
	.filter((v) => v !== '');

const outputDirs = [...vercelFuncDirs, LOCAL_BUILD_PATH];

for (const outputDir of outputDirs) {
	// We have to explicitly copy the WASM files to the output directory of each serverless function because Sveltekit + Vercel have no way of knowing that they need to be copied via static analysis.
	console.log(`Copying WASM files to ${outputDir}.`);
	await $`cp node_modules/@ffmpeg.wasm/core-mt/dist/core.wasm ${outputDir}`;
	await $`cp node_modules/@ffmpeg.wasm/core-mt/dist/core.worker.js ${outputDir}/core.worker.cjs`;
	await $`cp -r ./src/server/transcripts/audio ${outputDir}`;
}

Everyone is saying "simply reference the file in your script!" but it doesn't work. I've tried so many combos of Path.join() and fs.readFile

[eltigerchino]

Everyone is saying "simply reference the file in your script!" but it doesn't work. I've tried so many combos of Path.join() and fs.readFile

This wouldn’t work because the files still need to be copied specifically to the serverless function. The adapter isn’t doing this at the moment.

I have a PR ready but my only concern is that it copies all referenced assets such as images and fonts, etc that end up in the serverless function bundle, increasing the size (maybe hitting limits? Slowing down cold starts?). Maybe there should be a way to filter the files included? #10979

[eltigerchino]

closed by #10979

[eltigerchino]

Reopening because there still some cleanup to do

[Rich-Harris]

In 2.4.0 we added a new $app/server module with a read function that allows you to solve this problem.

• https://sveltekit-pdf-demo.vercel.app
• https://github.com/Rich-Harris/sveltekit-pdf-demo

Note that the demo uses pdf-lib rather than pdfmake. AFAICT pdfmake requires you to pass a filepath rather than an array buffer, which isn't ideal since we want to be as runtime-agnostic as possible. Maybe we could add a function for getting the filepath for an asset to solve those use cases but it wouldn't make sense in non-Node-like environments (whereas we intend for read to eventually be supported as widely as possible).

For now I'll treat this issue as fixed. Thanks!

[wesbos]

@eltigerchino @Rich-Harris so we can now simply reference the file and it will be included in the build? We have a dependency that just assumes the .wasm file exists alongside it, no way to pass it in, so we have to do this post-build pre-deploy script: https://github.com/syntaxfm/website/blob/main/why_do_i_need_this.mjs#L14

[eltigerchino]

@wesbos that’s exactly it! This excellent PR #11649 implements the file copying of server assets for the vercel adapter. I haven’t had the chance to try it out myself but you should be able to verify if the wasm files are in the build output of the serverless function