Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support content collections with % in filename #8684

Merged
merged 6 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/quick-wolves-drive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'astro': patch
---

Support content collections with % in filename
3 changes: 2 additions & 1 deletion packages/astro/src/content/types-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,8 @@ export async function createContentTypesGenerator({
});

for (const entry of globResult) {
const entryURL = new URL(entry.path, contentPaths.contentDir);
const fullPath = path.join(fileURLToPath(contentPaths.contentDir), entry.path);
const entryURL = pathToFileURL(fullPath);
if (entryURL.href.startsWith(contentPaths.config.url.href)) continue;
if (entry.dirent.isFile()) {
events.push({
Expand Down
16 changes: 14 additions & 2 deletions packages/astro/src/core/build/static-build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import glob from 'fast-glob';
import { bgGreen, bgMagenta, black, dim } from 'kleur/colors';
import fs from 'node:fs';
import path, { extname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { fileURLToPath, pathToFileURL } from 'node:url';
import * as vite from 'vite';
import type { RouteData } from '../../@types/astro.js';
import {
Expand Down Expand Up @@ -186,6 +186,17 @@ async function ssrBuild(
const sanitizedName = name.split('.')[0];
return `chunks/${sanitizedName}_[hash].mjs`;
}
// Detect if the chunk name has as % sign that is not encoded.
// This is borrowed from Node core: https://github.com/nodejs/node/blob/3838b579e44bf0c2db43171c3ce0da51eb6b05d5/lib/internal/url.js#L1382-L1391
// We do this because you cannot import a module with this character in it.
for(let i = 0; i < name.length; i++) {
if(name[i] === '%') {
const third = name.codePointAt(i + 2)! | 0x20;
if (name[i + 1] !== '2' || third !== 102) {
return `chunks/${name.replace(/%/g, '_percent_')}_[hash].mjs`;
}
}
}
Comment on lines +189 to +199
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like it should be possible to import modules with % but it needs to be escaped before importing. Is it happening inside the Rollup-generated code, or are we importing it wrongly?

Copy link
Contributor Author

@matthewp matthewp Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the rollup generated code, it has this exactly line import('../chunks/four%.hash.mjs') and Node cannot handle that. I think this might actually be a Node.js bug; not being able to import valid filename seems like a bug to me. But any event, if we escape it in the filename then everything works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked offline and this is either a Node.js bug or a Rollup bug, so going with the workaround for now. I'll be filing a bug with Node.js to see what they think.

return `chunks/[name]_[hash].mjs`;
},
assetFileNames: `${settings.config.build.assets}/[name].[hash][extname]`,
Expand Down Expand Up @@ -313,7 +324,8 @@ async function runPostBuildHooks(
? build.server
: build.client
: config.outDir;
const fileURL = new URL(fileName, root);
const fullPath = path.join(fileURLToPath(root), fileName);
const fileURL = pathToFileURL(fullPath);
await fs.promises.mkdir(new URL('./', fileURL), { recursive: true });
await fs.promises.writeFile(fileURL, mutation.code, 'utf-8');
}
Expand Down
6 changes: 4 additions & 2 deletions packages/astro/src/vite-plugin-markdown/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import {
import matter from 'gray-matter';
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { fileURLToPath, pathToFileURL } from 'node:url';
import type { Plugin } from 'vite';
import { normalizePath } from 'vite';
import type { AstroSettings } from '../@types/astro.js';
Expand Down Expand Up @@ -76,9 +76,11 @@ export default function markdown({ settings, logger }: AstroPluginOptions): Plug
const rawFile = await fs.promises.readFile(fileId, 'utf-8');
const raw = safeMatter(rawFile, id);

const fileURL = pathToFileURL(fileId);

const renderResult = await processor
.render(raw.content, {
fileURL: new URL(`file://${fileId}`),
fileURL,
frontmatter: raw.data,
})
.catch((err) => {
Expand Down
3 changes: 2 additions & 1 deletion packages/astro/test/content-collections.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,13 @@ describe('Content Collections', () => {

const ids = json.withSchemaConfig.map((item) => item.id);
const publishedDates = json.withSchemaConfig.map((item) => item.data.publishedAt);
expect(ids).to.deep.equal(['one.md', 'three.md', 'two.md']);
expect(ids).to.deep.equal(['four%.md', 'one.md', 'three.md', 'two.md']);
expect(publishedDates.every((date) => date instanceof Date)).to.equal(
true,
'Not all publishedAt dates are Date objects'
);
expect(publishedDates.map((date) => date.toISOString())).to.deep.equal([
'2021-01-01T00:00:00.000Z',
'2021-01-01T00:00:00.000Z',
'2021-01-03T00:00:00.000Z',
'2021-01-02T00:00:00.000Z',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: Four
description: The forth page
lang: en
publishedAt: 2021-01-01
---

# It's the forth page, fancy!
Loading