Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(env): client/public variables #10848

Merged
merged 46 commits into from
Apr 26, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
c7fffe0
feat: add schema, types and envField
florian-lefebvre Apr 17, 2024
0e90c5d
feat: move things around
florian-lefebvre Apr 18, 2024
94d6ffc
feat: add validators
florian-lefebvre Apr 19, 2024
d717361
feat: rework everything
florian-lefebvre Apr 20, 2024
d9ce895
feat: default
florian-lefebvre Apr 20, 2024
f136b1b
chore: space
florian-lefebvre Apr 20, 2024
0260bbb
Merge branch 'feat/astro-env-config' into feat/astro-env-validators
florian-lefebvre Apr 20, 2024
b84cea9
feat: reuse types
florian-lefebvre Apr 20, 2024
3f32ea6
feat: work on test
florian-lefebvre Apr 20, 2024
ff99f0d
feat: write test
florian-lefebvre Apr 20, 2024
8896fa7
feat: add test
florian-lefebvre Apr 20, 2024
9d8d425
Merge branch 'feat/astro-env-config' into feat/astro-env-validators
florian-lefebvre Apr 20, 2024
07f586e
feat: add vite plugin
florian-lefebvre Apr 20, 2024
5063b1f
feat: update zod custom validation
florian-lefebvre Apr 21, 2024
b23233c
feat: validation logic
florian-lefebvre Apr 21, 2024
e02b722
feat: work on types injection
florian-lefebvre Apr 21, 2024
e9d99a1
Merge branch 'feat/astro-env' into feat/astro-env-vite-plugin
florian-lefebvre Apr 22, 2024
bff9024
fix: test
florian-lefebvre Apr 22, 2024
fc983d1
Merge branch 'feat/astro-env-vite-plugin' into feat/astro-env-client-…
florian-lefebvre Apr 22, 2024
518fd15
fix: test
florian-lefebvre Apr 22, 2024
eeda833
fix: test
florian-lefebvre Apr 22, 2024
9516c2c
fix: cli tests
florian-lefebvre Apr 22, 2024
f23296d
feat: make things work
florian-lefebvre Apr 22, 2024
37ec524
feat: address todos
florian-lefebvre Apr 22, 2024
ae92463
Discard changes to examples/basics/astro.config.mjs
florian-lefebvre Apr 22, 2024
dc92fb6
Discard changes to examples/basics/src/env.d.ts
florian-lefebvre Apr 22, 2024
4496bc9
Merge branch 'feat/astro-env' into feat/astro-env-client-public
florian-lefebvre Apr 23, 2024
f6a993a
fix: test
florian-lefebvre Apr 23, 2024
1384136
feat: refactor astro sync test
florian-lefebvre Apr 23, 2024
26e2b82
feat: update test to check env types
florian-lefebvre Apr 23, 2024
f4b09b8
fix: fixture
florian-lefebvre Apr 23, 2024
c1539ed
feat: simplify injected types
florian-lefebvre Apr 24, 2024
4b5c28c
feat: move plugin
florian-lefebvre Apr 24, 2024
0c99784
fix: imports
florian-lefebvre Apr 24, 2024
20875e7
feat: add tests for getType
florian-lefebvre Apr 24, 2024
82db951
feat: add test
florian-lefebvre Apr 24, 2024
e093397
Update index.ts
florian-lefebvre Apr 25, 2024
770b806
fix: variable name conflict
florian-lefebvre Apr 25, 2024
e29e48d
Apply suggestions from code review
florian-lefebvre Apr 25, 2024
aa0312d
feat: harcode constants
florian-lefebvre Apr 25, 2024
353446e
feat: rename condition to meetsCondition
florian-lefebvre Apr 25, 2024
82a3142
feat: rename function
florian-lefebvre Apr 25, 2024
4e37b39
feat: simplify content/dts computation
florian-lefebvre Apr 25, 2024
5cb7e59
feat: throw one error with all invalid variables
florian-lefebvre Apr 25, 2024
5f12f8d
feat: refactor for clarity and prepare server/public
florian-lefebvre Apr 25, 2024
e519f82
feat: use more vite hooks
florian-lefebvre Apr 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions packages/astro/config.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ type ViteUserConfigFn = import('vite').UserConfigFn;
type AstroUserConfig = import('./dist/@types/astro.js').AstroUserConfig;
type ImageServiceConfig = import('./dist/@types/astro.js').ImageServiceConfig;
type SharpImageServiceConfig = import('./dist/assets/services/sharp.js').SharpImageServiceConfig;
type EnvField = typeof import('./dist/env/config.js').envField;

/**
* See the full Astro Configuration API Documentation
Expand Down Expand Up @@ -33,3 +34,8 @@ export function squooshImageService(): ImageServiceConfig;
* See: https://docs.astro.build/en/guides/images/#configure-no-op-passthrough-service
*/
export function passthroughImageService(): ImageServiceConfig;

/**
* TODO:
*/
export const envField: EnvField;
1 change: 1 addition & 0 deletions packages/astro/config.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export { defineConfig, getViteConfig } from './dist/config/index.js';
export { envField } from './dist/env/config.js';

export function sharpImageService(config = {}) {
return {
Expand Down
8 changes: 4 additions & 4 deletions packages/astro/src/@types/astro.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ import type {
} from '../transitions/events.js';
import type { DeepPartial, OmitIndexSignature, Simplify } from '../type-utils.js';
import type { SUPPORTED_MARKDOWN_FILE_EXTENSIONS } from './../core/constants.js';
import type { EnvSchema } from '../env/schema.js'
import type { EnvSchema } from '../env/schema.js';

export { type AstroIntegrationLogger };

Expand Down Expand Up @@ -1930,7 +1930,6 @@ export interface AstroUserConfig {
* TODO:
*/
env?: {

/**
* @docs
* @name experimental.env.schema
Expand All @@ -1941,8 +1940,8 @@ export interface AstroUserConfig {
*
* TODO:
*/
schema?: EnvSchema
}
schema?: EnvSchema;
};
};
}

Expand Down Expand Up @@ -2132,6 +2131,7 @@ export interface AstroSettings {
tsConfigPath: string | undefined;
watchFiles: string[];
timer: AstroTimer;
dotAstroDir: URL;
}

export type AsyncRendererComponentFn<U> = (
Expand Down
1 change: 0 additions & 1 deletion packages/astro/src/config/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import type { UserConfig } from 'vite';
import type { AstroUserConfig } from '../@types/astro.js';
import { Logger } from '../core/logger/core.js';
export { envField } from "../env/config.js";

export function defineConfig(config: AstroUserConfig) {
return config;
Expand Down
1 change: 0 additions & 1 deletion packages/astro/src/content/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ export { createContentTypesGenerator } from './types-generator.js';
export {
contentObservable,
getContentPaths,
getDotAstroTypeReference,
hasAssetPropagationFlag,
} from './utils.js';
export { astroContentAssetPropagationPlugin } from './vite-plugin-content-assets.js';
Expand Down
10 changes: 5 additions & 5 deletions packages/astro/src/content/types-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ async function writeContentFiles({
let contentTypesStr = '';
let dataTypesStr = '';

const collectionSchemasDir = new URL('./collections/', contentPaths.cacheDir);
const collectionSchemasDir = new URL('./collections/', settings.dotAstroDir);
if (
settings.config.experimental.contentCollectionJsonSchema &&
!fs.existsSync(collectionSchemasDir)
Expand Down Expand Up @@ -490,12 +490,12 @@ async function writeContentFiles({
}
}

if (!fs.existsSync(contentPaths.cacheDir)) {
fs.mkdirSync(contentPaths.cacheDir, { recursive: true });
if (!fs.existsSync(settings.dotAstroDir)) {
fs.mkdirSync(settings.dotAstroDir, { recursive: true });
}

const configPathRelativeToCacheDir = normalizeConfigPath(
contentPaths.cacheDir.pathname,
settings.dotAstroDir.pathname,
contentPaths.config.url.pathname
);

Expand All @@ -512,7 +512,7 @@ async function writeContentFiles({
);

await fs.promises.writeFile(
new URL(CONTENT_TYPES_FILE, contentPaths.cacheDir),
new URL(CONTENT_TYPES_FILE, settings.dotAstroDir),
typeTemplateContent
);
}
13 changes: 1 addition & 12 deletions packages/astro/src/content/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,6 @@ export const collectionConfigParser = z.union([
}),
]);

export function getDotAstroTypeReference({ root, srcDir }: { root: URL; srcDir: URL }) {
const { cacheDir } = getContentPaths({ root, srcDir });
const contentTypesRelativeToSrcDir = normalizePath(
path.relative(fileURLToPath(srcDir), fileURLToPath(new URL(CONTENT_TYPES_FILE, cacheDir)))
);

return `/// <reference path=${JSON.stringify(contentTypesRelativeToSrcDir)} />`;
}

export const contentConfigParser = z.object({
collections: z.record(collectionConfigParser),
});
Expand Down Expand Up @@ -430,7 +421,6 @@ export function contentObservable(initialCtx: ContentCtx): ContentObservable {
export type ContentPaths = {
contentDir: URL;
assetsDir: URL;
cacheDir: URL;
typesTemplate: URL;
virtualModTemplate: URL;
config: {
Expand All @@ -440,13 +430,12 @@ export type ContentPaths = {
};

export function getContentPaths(
{ srcDir, root }: Pick<AstroConfig, 'root' | 'srcDir'>,
{ srcDir }: Pick<AstroConfig, 'root' | 'srcDir'>,
fs: typeof fsMod = fsMod
): ContentPaths {
const configStats = search(fs, srcDir);
const pkgBase = new URL('../../', import.meta.url);
return {
cacheDir: new URL('.astro/', root),
contentDir: new URL('./content/', srcDir),
assetsDir: new URL('./assets/', srcDir),
typesTemplate: new URL('content-types.template.d.ts', pkgBase),
Expand Down
1 change: 1 addition & 0 deletions packages/astro/src/core/config/settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ export function createBaseSettings(config: AstroConfig): AstroSettings {
watchFiles: [],
devToolbarApps: [],
timer: new AstroTimer(),
dotAstroDir: new URL('.astro/', config.root),
};
}

Expand Down
2 changes: 1 addition & 1 deletion packages/astro/src/core/create-vite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ export async function createVite(
// the build to run very slow as the filewatcher is triggered often.
mode !== 'build' && vitePluginAstroServer({ settings, logger, fs }),
envVitePlugin({ settings }),
astroEnvVirtualModPlugin({ settings, logger, mode, fs }),
markdownVitePlugin({ settings, logger }),
htmlVitePlugin(),
mdxVitePlugin(),
Expand All @@ -156,7 +157,6 @@ export async function createVite(
astroDevToolbar({ settings, logger }),
vitePluginFileURL({}),
astroInternationalization({ settings }),
astroEnvVirtualModPlugin({ settings, logger })
],
publicDir: fileURLToPath(settings.config.publicDir),
root: fileURLToPath(settings.config.root),
Expand Down
12 changes: 12 additions & 0 deletions packages/astro/src/core/errors/errors-data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1156,6 +1156,18 @@ export const i18nNotEnabled = {
hint: 'See https://docs.astro.build/en/guides/internationalization for a guide on setting up i18n.',
} satisfies ErrorData;

/**
* @docs
* @description
* The failing environment variable does not match the type and constraints your defined in `experimental.env.schema`.
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
*/
export const EnvInvalidVariable = {
name: 'EnvInvalidVariable',
title: 'Invalid Environment variable',
message: (key: string, type: string) => `Variable "${key}" is not of type: ${type}.`,
hint: 'The failing environment variable does not match the type and constraints your defined in `experimental.env.schema`.',
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
} satisfies ErrorData;

/**
* @docs
* @kind heading
Expand Down
11 changes: 11 additions & 0 deletions packages/astro/src/env/constants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
function resolveVirtualModuleId<T extends string>(id: T): `\0${T}` {
return `\0${id}`;
}

export const VIRTUAL_CLIENT_MODULE_ID = 'astro:env/client';
export const RESOLVED_VIRTUAL_CLIENT_MODULE_ID = resolveVirtualModuleId(VIRTUAL_CLIENT_MODULE_ID);
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
export const VIRTUAL_SERVER_MODULE_ID = 'astro:env/server';
export const RESOLVED_VIRTUAL_SERVER_MODULE_ID = resolveVirtualModuleId(VIRTUAL_SERVER_MODULE_ID);

export const PUBLIC_PREFIX = 'PUBLIC_';
export const ENV_TYPES_FILE = 'astro-env.d.ts';
2 changes: 1 addition & 1 deletion packages/astro/src/env/schema.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { z } from 'zod';
import { PUBLIC_PREFIX } from './constants.js';

const StringSchema = z.object({
type: z.literal('string'),
Expand Down Expand Up @@ -33,7 +34,6 @@ const SecretServerEnvFieldMetadata = z.object({
});

const KEY_REGEX = /^[A-Z_]+$/;
const PUBLIC_PREFIX = 'PUBLIC_';

export const EnvSchema = z
.record(
Expand Down
18 changes: 11 additions & 7 deletions packages/astro/src/env/validators.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@ export type ValidationResultValue = EnvFieldType['default'];
type ValidationResult =
| {
ok: true;
type: string;
value: ValidationResultValue;
}
| {
ok: false;
error: string;
type: string;
};

const errorMsg = (key: string, options: EnvFieldType) => {
const optional = options.optional ?? options.default;
return `Variable "${key}" is not of type: ${options.type}${optional ? '| undefined' : ''}.`;
};
export function getType(options: EnvFieldType) {
const optional = options.optional ? (options.default !== undefined ? false : true) : false;
return `${options.type}${optional ? ' | undefined' : ''}`;
}

type ValueValidator = (input: string | undefined) => {
valid: boolean;
Expand Down Expand Up @@ -46,7 +47,6 @@ const booleanValidator: ValueValidator = (input) => {
};

export function validateEnvVariable(
key: string,
value: string | undefined,
options: EnvFieldType
): ValidationResult {
Expand All @@ -56,11 +56,14 @@ export function validateEnvVariable(
boolean: booleanValidator,
}[options.type];

const type = getType(options);

if (options.optional || options.default !== undefined) {
if (value === undefined) {
return {
ok: true,
value: options.default,
type,
};
}
}
Expand All @@ -69,10 +72,11 @@ export function validateEnvVariable(
return {
ok: true,
value: parsed,
type,
};
}
return {
ok: false,
error: errorMsg(key, options),
type,
};
}
95 changes: 92 additions & 3 deletions packages/astro/src/env/vite-plugin-env-virtual-mod.ts
Original file line number Diff line number Diff line change
@@ -1,28 +1,117 @@
import type { Plugin } from 'vite';
import { loadEnv, type Plugin } from 'vite';
import type { AstroSettings } from '../@types/astro.js';
import type { Logger } from '../core/logger/core.js';
import {
ENV_TYPES_FILE,
RESOLVED_VIRTUAL_CLIENT_MODULE_ID,
VIRTUAL_CLIENT_MODULE_ID,
} from './constants.js';
import type { EnvSchema } from './schema.js';
import { validateEnvVariable } from './validators.js';
import { AstroError, AstroErrorData } from '../core/errors/index.js';
import type fsMod from 'node:fs';
import { fileURLToPath } from 'node:url';

interface AstroEnvVirtualModPluginParams {
settings: AstroSettings;
logger: Logger;
mode: 'dev' | 'build' | string;
fs: typeof fsMod;
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
}

export function astroEnvVirtualModPlugin({
settings,
logger,
mode,
fs,
}: AstroEnvVirtualModPluginParams): Plugin | undefined {
if (!settings.config.experimental.env) {
return;
}

logger.warn('env', 'This feature is experimental. TODO:');

const { schema } = settings.config.experimental.env;
// TODO: client / public
const schema = settings.config.experimental.env.schema ?? {};
const loadedEnv = loadEnv(
mode === 'dev' ? 'development' : 'production',
fileURLToPath(settings.config.root),
''
);

const { clientContent, clientDts } = handleClientModule({ schema, loadedEnv });
handleDts({ settings, fs, content: clientDts });

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are generating code before the load function; is that correct? Shouldn't we generate code inside load? Types and client code will be emitted only if the user attempts to use the virtual module.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I also think we should be lazy loading the virtual module code. One thing to also check though is if new env vars get picked up across server restarts. As a Vite plugin, it's best to cache or reset the cache with the buildStart/buildEnd/config/configResolved hooks etc so that we re-compute the env when needed.

// TODO: server / public
// TODO: server / secret
return {
name: 'astro-env-virtual-mod-plugin',
enforce: 'pre',
resolveId(id) {
if (id === VIRTUAL_CLIENT_MODULE_ID) {
return RESOLVED_VIRTUAL_CLIENT_MODULE_ID;
}
},
load(id) {
if (id === RESOLVED_VIRTUAL_CLIENT_MODULE_ID) {
return clientContent;
}
},
};
}

function handleDts({
content,
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
settings,
fs,
}: {
content: string;
settings: AstroSettings;
fs: typeof fsMod;
}) {
fs.mkdirSync(settings.dotAstroDir, { recursive: true });
fs.writeFileSync(new URL(ENV_TYPES_FILE, settings.dotAstroDir), content, 'utf-8');
}
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved

function handleClientModule({
schema,
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
loadedEnv,
}: {
schema: EnvSchema;
loadedEnv: Record<string, string>;
}) {
const data: Array<{ key: string; value: any; type: string }> = [];

for (const [key, options] of Object.entries(schema)) {
if (options.context !== 'client') {
continue;
}
const variable = loadedEnv[key];
const result = validateEnvVariable(variable === '' ? undefined : variable, options);
if (!result.ok) {
throw new AstroError({
...AstroErrorData.EnvInvalidVariable,
message: AstroErrorData.EnvInvalidVariable.message(key, result.type),
});
}
data.push({ key, value: result.value, type: result.type });
florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
}

const clientContent = `
const data = ${JSON.stringify(Object.fromEntries(data.map((e) => [e.key, e.value])))};

${data.map((e) => `const ${e.key} = data.${e.key};`).join('\n')}

export {
${data.map((e) => e.key).join(',\n')}
}
`;

const clientDts = `declare module "astro:env/client" {
${data.map((e) => `export const ${e.key}: ${e.type};`).join('\n ')}
}`;

florian-lefebvre marked this conversation as resolved.
Show resolved Hide resolved
return {
clientContent,
clientDts,
};
}
Loading
Loading