Implement D1Database using D1JS and D1 API

[mrbbot]

Previously, Miniflare had its own implementations of BetaDatabase and Statement. These were subtly different to the implementations in the D1 Wrangler shim D1JS, causing behaviour mismatches.

This change switches the implementation to use the shim, with Miniflare implementing the underlying D1 HTTP API instead. We'll need to do this anyway when adding D1 support to Miniflare 3 using workerd.

Specific changes:

• Throw when calling D1PreparedStatement#run() with statements that return data, closes 🐛 BUG: .run() throws D1 Error in cloud #441
• Fix response envelope format, closes BUG: Different responses when running a D1 query in the cloud and the local mode #442 and 🐛 BUG: D1 Result inconsistency with docs, live, and wrangler dev workers-sdk#2504
• Fix binding/return of BLOB-typed values, closes 🐛 BUG: wrangler D1 not binding BLOB in the same format than cloud D1 workers-sdk#2527
• Fix D1Database#raw() return, closes 🐛 BUG: .raw() does not work locally workers-sdk#2238 (already fixed in Fix .raw() not executing statement in D1 #474)
• Add support for D1Database#dump()
• Run D1Database#{batch,exec}() statements in implicit transaction
• Only run first statement when calling D1PreparedStatement#{first,run,all,raw}()

[changeset-bot]

⚠️ No Changeset found

Latest commit: a54e8b7

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[geelen]

Ahh, so you're still expecting Wrangler not to shim the binding (see https://github.com/cloudflare/wrangler2/blob/main/packages/wrangler/templates/d1-beta-facade.js#L217-L219) but then you're having Miniflare use the shim code to generate the right API?

That should work, but you might like to consider changing Wrangler's behaviour to always inject the shim, and then Miniflare's D1 only needs to export the D1DatabaseAPI with its .fetch method.

I'm not sure if your testing story would be worse in that case, though, so maybe it's better to keep it as it is... In fact, we do want the Wrangler shim to disappear eventually (which is why I built the Miniflare stuff the way I did), so this is probably the right approach.

Approved.

[mrbbot]

I'm not sure if your testing story would be worse in that case

Yeah, the reason I did it this way is so users can continue to run Miniflare standalone (with Miniflare's CLI or in the testing environments) without having to bundle with Wrangler first. This would be a breaking change otherwise.

For Miniflare 3, we'll probably switch to Wrangler doing the shimming, or if you're planning to remove that, none at all. 👍

[ruslantalpa]

@mrbbot
I think this PR broke queries like select ?1 + ?2

statements.ts used to have this code

      // For statements using ?1 ?2, etc, we want to pass them as varargs but
      // "better" sqlite3 wants them as an object of {1: params[0], 2: params[1], ...}
      if (this.#bindings.length > 0 && typeof this.#bindings[0] !== "object") {
        return prepared.bind(
          Object.fromEntries(this.#bindings.map((v, i) => [i + 1, v]))
        );
      } else {
        throw e;
      }

[ruslantalpa]

i added these lines to one of the tests (to make sure it's not my code):

const sum = await db.prepare("SELECT ?1 + ?2 as sum").bind(1, 2).first("sum");
t.is(sum, 3);

and the test fails

d1 › test › d1js.ts › D1PreparedStatement: first

  packages/d1/test/d1js.spec.ts:212

   211:
   212:   const sum = await db.prepare("SELECT ?1 + ?2 as sum").bind(1, 2).first("sum");
   213:   t.is(sum, 3);

  Rejected promise returned by test. Reason:

  Error {
    message: 'D1_ERROR',
  }

  › D1Database._send (packages/d1/src/d1js.ts:148:13)
  › D1PreparedStatement.first (packages/d1/src/d1js.ts:215:7)
  › packages/d1/test/d1js.spec.ts:212:15

  ─