Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement Vitest testing environment for Miniflare 3 (#4795)
* [miniflare] Add support for module fallback service Adds a `unsafeUseModuleFallbackService` worker option and a `unsafeModuleFallbackService` shared option for using `workerd`'s module fallback service. Note `workerd` only accepts an address for fallback service configuration. This means if we wanted per-worker fallback service configuration, we'd need to start a new HTTP server for each worker that enabled it. To avoid starting additional servers, we reuse Miniflare's existing loopback server, and allow each worker to enable/disable the fallback service instead. * [miniflare] Return `Response` from `fetch()` if socket upgrade failed Previously, Miniflare would throw an error if the `Upgrade: websocket` header was set, but a non-WebSocket response was returned. This change ensures the response is returned, just without `webSocket` set. This allows the caller to handle the error case/response themselves. * MVP for `@cloudflare/vitest-pool-workers` Implements a basic Vitest pool that runs tests inside `workerd`, using the module fallback service and a test runner Durable Object. Provides access to bindings via `env` from the `cloudflare:test` module. Closes DEVX-1052 and DEVX-1053 * Implement workspace/test file worker isolation Adds a `singleWorker` option for running all a project's test files serially in the same worker. This can speed up execution if the project contains lots of small test files, as Vitest only needs to be initialised once. * Add Durable Object test helpers and support for self-service bindings Adds a `runInDurableObject()` test helper for running arbitrary code inside a Durable Object's I/O context, and a `runDurableObjectAlarm()` helper for running a Durable Object's scheduled alarms. These require the new `main` option to be set to the worker's entrypoint, so the Vitest pool can create instances of the user's Durable Objects. The `main` option also allows service bindings to be declared to the worker running the tests. Events dispatched to these service bindings will use the handler exported by the `main` worker. Closes DEVX-1056, DEVX-1057 and DEVX-1058 * Add support for fetch mocking with `undici` `MockAgent` Adds `fetchMock` to the `cloudflare:test` module. This is an instance of `undici`'s `MockAgent`, and can be used to mock global `fetch()` requests in the test worker. This works by bundling just the `undici` `MockAgent` code, with a customisable `Dispatcher` that is set to the original workers `fetch()` function in the pool. Closes DEVX-1059 * Use custom test runner and reset fetch mock at the start of each file A custom test runner allows us to hook into test execution. Specifically, this allows us to run code before and after files/suites/tests are executed. We'll need this for isolated storage. * Add (inline) snapshot support and fix source maps By default, Vitest uses the `node:fs` module for reading/writing snapshots. This isn't available in workers, so this change implements a custom snapshot client using service bindings. Closes DEVX-1054 * Fix logs not showing on re-runs * [miniflare] Export `formatZodError` We'd like to use Zod for validating the pool's options. Miniflare has a `formatZodError` function that makes pretty error messages from Zod validation errors. * Validate pool options with Zod and pretty error formatting * Bump to stable `vitest@1.0.1` * Add support for serialising any structured serialisable * Add support for fake timers * Verify module mocking and spies work * [miniflare] Add support for "sticky blobs" that aren't deleted For isolated per-test storage, we'll be pushing/popping databases to a "stack". After pushing a database stack frame, we may delete a record in a test. We'd like to avoid deleting the associated blob here so when we pop the frame, the record still links up with it. This change adds an `unsafeStickyBlobs` option that disables garbage blob cleanup if enabled. * [miniflare] Add support for getting persistence paths Isolated storage will need access to SQLite database files. This change adds a `unsafeGetPersistPaths()` function for getting the persistence paths used by a `Miniflare` instance. This is unsafe as the layout of persistence directories is not a public interface. * [miniflare] Add support for colo-local ephemeral "Durable" Objects For isolated storage, we'd like the runner object not to persist any state to disk, so we don't have to worry about excluding it when managing the storage stack. We currently have an `unsafeEphemeralDurableObjects` option that makes _all_ Durable Objects in a worker use in-memory state. This isn't suitable for our Vitest pool, as user objects will run in the same worker as the runner object. This change adds support for `workerd` "colo-local" actors. These are like Durable Objects, but don't provide any durable storage (i.e. no `state.storage` API), and have a slightly different namespace binding API. * [miniflare] Bind `this` to `Miniflare` instance in custom services We'd like to extend our Vitest pool's custom loopback service with endpoints for managing isolated storage stacks. To do this, we need to know the persistence paths of the `Miniflare` instance we're currently running in. A previous commit added an `unsafeGetPersistPaths()` function. If we wanted to access these paths in the pool's loopback service, we'd need to capture the paths/`Miniflare` instance in a closure used as the service binding function. Unfortunately, we use `util.deepStrictEqual()` on Miniflare options to determine if we should restart the `Miniflare` instance. This would always return `true` if we created a new bound/arrow function on each test run. This change binds `this` to the current `Miniflare` instance when calling custom service functions, meaning we can reuse the same function when building options. * Add support for isolated storage This change adds an `isolatedStorage` pool option. If enabled, each test gets its own isolated storage environment. Storage writes in a test are automatically undone at the end of the test. The storage environment is copied from the parent suite, ensuring writes from `beforeAll()`s show up in tests. This is implemented with an on-disk stack of `.sqlite` files. Before a suite/test attempt begins, the current storage state is "pushed". When the suite/test attempt completes/fails, the state is "popped", undoing all changes. Sticky blobs are used to ensure deleting records in a test keeps the corresponding blobs around, so they can be used when the state is "popped". Closes DEVX-1055 * [miniflare] Use `z.input()` for user facing options types Previously, we were using `z.infer()` on our option schemas to derive `WorkerOptions` and `SharedOptions` types. This gave the _output_ types for schemas, rather than the user _input_ types. Usually, these were the same. However, if a schema was `.transform()`ed, they could be different. Notably, `hyperdrives` values output objects, but accept strings, leading to type errors if these were used. * Add `defineWorkersPoolOptions()` for typed pool options in config * [miniflare] Easy service bindings to self * Use Miniflare's mechanism for self-service bindings * Tidy up Node and package polyfills Tries to remove unnecessary polyfills/functions, so users are less likely to import APIs that would be unsupported in production in their tests. * Throw actual unhandled errors in pool on failed test runs Tunnels through the error thrown when running tests fails. This is useful for debugging, as it means the startup error thrown by Vitest includes a useful message and stack. * Add argument validation to `cloudflare:test` Durable Object helpers While we expect most users to write their tests in TypeScript, JavaScript tests are also supported. Since we can't guarantee correct types at runtime, this change adds type validation to test helpers, throwing errors in the same JSG-style as other `workerd` APIs. * [miniflare] Add support for `rootPath`s Previously, Miniflare would always resolve path-valued options relative to the current working directory. For the Vitest pool, we'd like to be able to resolve these relative to the project config file. This change adds back Miniflare 2's `rootPath` option for controlling the resolution directory. `rootPath` can be set as a shared or per-worker option. Path-valued options resolve relative to the closest `rootPath`. * [miniflare] Fix linting scripts * Resolve pool Miniflare options relative to `vitest.config.ts` ...rather than the current working directory. This ensures paths are always resolved to the same location when `vitest` is run in different directories. * Support injecting provided values in pool options Vitest global setup hooks provide a good place to start auxiliary servers required by tests (e.g. upstream). This change allows provided values to be injected into the Miniflare config (e.g. using port from global setup in hyperdrive local connection string). * Support seeding data in config and reset storage between runs This change adds a `setupEnvironment(env)` function called with a bindings proxy per-`Miniflare`-instance per-test-run. This should be used for seeding data from disk (e.g. applying D1 migrations). * [miniflare] Fix `DurableObjectStub` detection Enabling Durable Object JS RPC changes the class name of `DurableObjectStub` from `DurableObject` to `WorkerRpc`. This was breaking `Fetcher#fetch()` detection and WebSocket-upgrading `DurableObjectStub#fetch()`es when using the magic proxy. * Bump to `vitest@1.1.3` * Support calling `fetch`/`scheduled`/`queue` handlers directly This change adds some new test helpers useful when importing Worker source code and calling handlers directly. Namely... - `createExecutionContext()` for creating an `ExecutionContext` - `getWaitUntil()` for waiting for all `ctx.waitUntil()` callbacks - `createScheduledController()` for creating a `scheduled` handler's `ScheduledController` - `getScheduledResult()` for getting the "no-retry" state of a `ScheduledController` - `createMessageBatch()` for creating a `queue` handler's `MessageBatch` - `getQueueResult()` for getting ack/retry state of a `MessageBatch` Note `createScheduledController()`/`getScheduledResult()` and `createMessageBatch()`/`getQueueResult()` take/return the same types as `Fetcher#scheduled()` and `Fetcher#queue()` respectively. Closes DEVX-1060 * Block custom environments Our Vitest pool runs tests inside `workerd` itself. This means we don't need to use Vitest's custom environment feature to setup the correct global scope. Modifying the global scope would result in some franken-`workerd`-environment that didn't match production. With Miniflare 2's `vitest-environment-miniflare`, we _did_ require users to enable a custom environment. Considering this is a significant difference, this change adds a very helpful error message if we detect a custom `environment` is configured. * Assert compatible Vitest version on startup Our Vitest pool currently depends on internal Vitest APIs that are not protected by semantic-versioning guarantees. Therefore, this change adds a runtime sanity check for a peer-dependency version constraint on `vitest`. * Fix fallback service on Windows * [miniflare] Update API docs for unsupported products * Integrate `vitest-pool-workers` with monorepo tooling * Only include `dist` directory when publishing * [pre] Move `CloudflareTestEnv` to `cloudflare:test` `ProvidedEnv` * [pre] Use correct project path for non-workspace projects `WorkspaceProject#path` was actually giving the path to the directory containing the Vitest config, not the config file itself, for non-workspace projects. * [pre] Manually construct `__dirname`s `vite-node` was transforming the pool files during development, but externalising them when the pool was depended on externally. When Vite transforms files, it automatically injects `__dirname`, even though these are ES modules, which shouldn't have this by default. * [pre] Construct serialised `poolOptions` to prevent serialising symbol * [pre] Include `cloudflare:test` types in package These can be referenced by adding `@cloudflare/vitest-pool-workers` to the `types` array in `tsconfig.json`. * Fixup comments: move method comments to JSDocs & add additional docs * fixup! [miniflare] Add support for `rootPath`s * Allow pool restarts on config change Vitest will dispose of the pool and recreate it when the Vitest config changes. This was causing `Miniflare` instances to be disposed, then reused when the pool was recreated. This change moves the project cache from the module-level to a "pool-level" local variable. * Get builtin module list and `ExportedHandler` keys from RTTI * fixup! Add support for fetch mocking with `undici` `MockAgent` Our mock agent implementation depends on `undici` internals. This change ensures we pin our version, so we don't accidentally upgrade to an incompatible version. * fixup! [miniflare] Bind `this` to `Miniflare` instance in custom services Switch to passing `Miniflare` instance as parameter instead of `this` * Use the same `esbuild` version as `wrangler` * Use `node:path/posix` for posix path helpers in module fallback * fixup! Fix fallback service on Windows * Check all parents in `"type": "module"` cache before reading files * fixup! Support seeding data in config and reset storage between runs Rename abort-all helper worker * Remove unnecessary `if` statement in fallback service * fixup! Tidy up Node and package polyfills Update not-implemented messages * Remove unnecessarily complex `groupBy()` * Add back support for listing Durable Objects in a namespace Adds a `listDurableObjectIds()` test helper that behaves similarly to `getMiniflareDurableObjectIds()` from Miniflare 2's test environments. * [miniflare] Allow `URL`s to be passed in `hyperdrives` option Importantly, this change makes the Zod output type of `hyperdrives` assignable to the input type. This means we can pass the output of parsing options schemas back to the `new Miniflare()` constructor. Also adds a few more `hyperdrives` tests to verify this doesn't change behaviour. * Remove `setupEnvironment()` in favor of services to access file-system * Add `SELF` export to `cloudflare:test` module Exposing `kCurrentWorker` to users for integration testing wasn't very nice. This is a common use case so this change ensures it's always bound and adds it directly to the `cloudflare:test` module. * Replace `getWaitUntil` with `waitOnExecutionContext` and `void` return This function aims to allow users to wait on side-effecting `waitUntil`s, then assert on their effects. `workerd` ignores the resolved value of `Promise`s, so exposing these may make it trickier to change the implementation of this function in the future. * Remove redundant `isolateDurableObjectBindings` serialised option `durableObjectBindingDesignators` includes the same and more information, so there's no need to pass this. * Fail if required compatibility flags aren't set This makes it explicit which flags we're enabling, and helps avoid behaviour differences when running `wrangler dev` or deployed code. * Rename `WorkersProjectOptions` to `WorkersPoolOptions` * fixup! Add support for isolated storage Clarify `StackedStorageState` is per `Miniflare`-instance * fixup! Add `SELF` export to `cloudflare:test` module Update docs in `config.ts` * Format with monorepo's `prettier` version
- Loading branch information
