The v3 release & continued support for v2.

[SupremeTechnopriest]

@mrbbot Are you planning to continue support for v2? The switch to workerd kind of removed everything that made miniflare great... Losing the get* methods is a real bummer. One example of where this hurts is setting up the dev environment. I have a seed file that I use to set up my locally persisted KV Stores. I tried to implement the same functionality using a dispatchFetch to a temporary worker that put the json request body in the kv store, but I couldn't get it to work. Testing also took a big hit in v3...

While having an identical runtime is nice (no more behavior mismatch label), losing all the tooling that made miniflare great is a questionable trade off. It was very rare that I ran into behavior mismatches, but now I cant unit test my code and storage persistence seems to be broken all together (.mf/v3/kv/unused?). The release of v3 leaves me with these questions:

• Are there plans to bring these missing features back to v3?
• If so is there a public roadmap/timeline we can follow?
• And in the meantime, will v2 still be maintained?

Hopefully the answer to all of these questions is yes because the v3 release feels incomplete and premature. Not trying to disparage all of your hard work. You have definitely made workers 1000000x better to work with and I appreciate you thoroughly. Its just that the v3 release was an emotional roller coaster. I saw a new version available, got super excited, started upgrading my project, was quickly disappointed, and had to roll back. It always feels icky running an outdated version of a dependency. 😢

[DaniFoldi]

Hi @SupremeTechnopriest 👋,

While I'm not MrBBot himself, I hope I can answer some of your questions so he can focus on - you guessed it - fixing some bugs and adding features into v3.

While I can't comment on promised support, v2 currently works for me for the majority of use cases that I need a local simulator for, and I doubt anything will break it in the next few months.

The reason some of these automated test utilities (get* methods) are more difficult to implement now is that your worker is running in workerd, while say vitest is running in node, two different processes. While I cannot guarantee anything, I know that you're not the only user who relied on these functions, and miss them from v3, so I wouldn't be surprised if they were added in some fashion in the near future.

Not sure what's gone wrong with storage persistence, the format is different to v2, so I recommend removing the folder entirely and let v3 recreate everything from scratch. If that doesn't fix persistence for you, opening an issue here with a minimal repro will help the team triage and fix it quickly.

To answer your second question as well (weird ordering, apologies), I don't know of any public roadmap for Miniflare, you can follow the repo, or join the Discord (#miniflare channel) to find out as new releases come out.

Hope this helps 😄

[mrbbot]

Hey! 👋

Thanks for starting this discussion.

Are you planning to continue support for v2?
And in the meantime, will v2 still be maintained?

I would love to, but I really don't think we have enough time/resources to. 😢 Our plan at the moment is to only provide security updates for Miniflare 2, until the next major version, at which point Miniflare 2 will be archived. It's very unlikely we'll add support for new bindings like Analytics Engine or Browsers to Miniflare 2. I know this kinda sucks, but unfortunately there's only so many hours in the day...

Losing the get* methods is a real bummer.

Definitely agree here, but again as @DaniFoldi points out, it's going to be tricky (but not necessarily impossible) to add these back. What we don't want to do is re-implement all the binding classes again in Node.js. This is what we were trying to avoid with Miniflare 3. What we could do is define some magic stub Proxys that forwarded method calls to a Worker with the appropriate bindings. The problem with this approach is that you end up going Node -> workerd -> Node and back. That being said, this is still true when seeding data with dispatchFetch() (assuming the seed data isn't defined in the Worker), so maybe it's just inevitable.

I tried to implement the same functionality using a dispatchFetch to a temporary worker that put the json request body in the kv store, but I couldn't get it to work.

If you could share the Worker/configuration that isn't working, that would be super helpful. It's possible there's a bug in Miniflare here we should fix. 🙂

Testing also took a big hit in v3...

Are you referring to the unit testing environments here? If so, I'd definitely like to bring these back. Again, it's much harder to implement these now with the runtime APIs being defined in a separate process.

.mf/v3/kv/unused?

Miniflare will now use the KV namespace ID as opposed to the binding name when persisting data. I suspect this is where the unused is coming from. Whilst it was previously unused in Miniflare 2, it's now used in Miniflare 3. This allows you to bind the same KV namespace in multiple Workers with different binding names.

Are there plans to bring these missing features back to v3?

• Step-through debugging: definitely
• Trigger scheduled events via CRON schedules: probably
• HTTPS servers: probably
• Unit testing environments: hopefully 😃
• getBindings()/getKVNamespace()/getR2Bucket()/getCaches()/getDurableObjectNamespace(): maybe, but they'd probably be deprecated to push people to using Workers directly
• getGlobalScope()/getModuleExports(): probably not, since these methods returned references to the actual global scope and module exports that you could modify

If so is there a public roadmap/timeline we can follow?

Not yet, but we could probably have a public roadmap for Miniflare. Will have to check. 👍

---

Thanks again for the feedback though, hopefully you can understand my perspective. 🙂

[SupremeTechnopriest]

Thank you both for your responses.

I definitely understand your perspective. For me personally, the most important feature is the unit testing environments. I have over 700 tests on our worker that allow me to sleep soundly at night. Our worker handles a substantial amount of mission critical traffic from fortune 500 e-commerce sites and without coverage every publish would be a nail biter. Second would be the get* methods. Mainly just to set up the dev environment and enhance unit testing. I dont really care how performant they are, I just need a way to interface with the stores. I thought about wrapping the API up in little workers (pseudo):

async function kvGet (namespace: string, key: string, type: 'text' | 'json' | 'arrayBuffer') {
  const mf = new Miniflare({
    script: `export default {
      async fetch(request, env, ctx) {
        const result = await env.${namespace}.get('${key}', type)
        return new Response(result)
      }
    }`,
    kvPersist: true,
    kvNamespaces: [namespace]
  })

  await mf.dispatchFetch()
  const res = await mf.dispatchFetch("http://localhost")

  switch (type) {
    case 'json':
      return res.json()
    case 'arrayBuffer':
      return res.arrayBuffer()
    case 'text':
      return res.text()
  }
}

const value = await kvGet('KV_STORE', 'myKey', 'json')

...but I couldn't get it to work. I will open an issue for this. The seed attempt is similar to the above, just with the put method. I attempt to parse the request body from the dispatchFetch and insert it into the kv store with persist enabled, but the record is not added to the file system.

The unused is definitely coming from the wrangler.toml. That makes perfect sense and is a nice refactor!

I have another worker that is less sensitive, and will probably start using minflare 3 there, but until unit testing is back, our main product is going to have to remain on miniflare 2. We are starting to integrate Browser bindings and TCP sockets, so I guess those features are going to be yolo for now.

Thanks again for your efforts. If there's anything I can do to help facilitate the return of the testing environments feel free to reach out.

[SupremeTechnopriest]

Another regret I have is that fetch response mocking will no longer work. You did such a nice job exposing the undici mock agent and it was a super helpful testing utility. 😢

[mrbbot]

This is a feature I think we should be able to bring back. workerd lets us customise the destination of outbound fetch() requests, and we should be able to build a similar API to undici's MockAgent for intercepting these. 👍

[SupremeTechnopriest]

Amazing!

[admah]

@SupremeTechnopriest PM for the DevProd team here 👋 I'm really sorry for the issues that you faced with the release. It's definitely not our intention to cause stress like that for you or any user. Thanks for your understanding and outlining the ways this change has affected your experience.

@mrbbot and I (along with the whole Workers DevProd team) will be working to sort out the various issues that you've raised in light of our long-term goal which is to increase the ability of users like yourself to test their Workers (and related resources) in every way possible, as early as possible.

To reiterate, we don't intend to stop supporting v2 in terms of security updates or major issues that might come up, but we want to focus our team's available energy for new development on v3.

Happy to schedule a time to discuss more if that's help!

[SupremeTechnopriest]

@admah Thanks for the follow up! I'm all for the direction you guys are moving in and understand the challenges. I wouldn't mind hopping in discord or zoom to chat about the details, but as long as v2 is going to be supported until v3 is as capable we should be fine. Its a bit of a disappointment to not have simulation of the new features in v2 (specifically browser and tcp sockets), but the decision makes sense. Hopefully unit testing support is the top priority. Once that is there I would feel comfortable making the switch.

Let me know if it would be helpful for you to hop on a call. I can give you some background on our use case. We have been on this journey with you since day 1 of workers. We were one of (if not the first) companies to use workers in production and the platform has come such a long way.

[GeoffreyBooth]

I’m using the method described in sveltejs/kit#2966 (comment) to simulate Cloudflare KV and DO for local development of a SvelteKit app. I too am concerned about the lack of migration path for this use case. SvelteKit seems to be a great pairing with Cloudflare Pages, as SvelteKit provides a first-party Cloudflare adapter; it would be a shame if there was no good story for local development with Cloudflare’s services.

[TimoWilhelm]

Same issue here! I'll have to stick with Miniflare v2 for the forseeable future if we can't access bindings as it's the only sane way to develop SvelteKit Apps using Cloudflare Platform features locally. Especially if a lot of state logic lives in Durable Objects or KV.

[jdddog]

IMHO, the most useful feature of Miniflare V2 that is absent in V3 are the unit testing environments, particularly the ability to write tests against various storage options without having to resort to mocking.

[SupremeTechnopriest]

Hey @admah,

Any progress on testing utilities? We just had our first global outage due to a mismatch in runtime between wrangler 2 and 3. I can no longer trust my tests... which is going to cause me to lose sleep...

[admah]

@SupremeTechnopriest yes, but still in planning stages. Our goal is to have something for unit testing by the end of the quarter or early next.

On a separate, but related note to this thread, we have been working on the following related to the Miniflare v3 release:

• added HTTPS support (Add https option to have miniflare accept https requests #612)
• added back support for fetchMock with the same Miniflare v2 API ([Miniflare 3] Add back support for fetchMock #632, [Miniflare 3] Add outboundService option for customising global fetch() target #629)
• added local support for Wrangler KV and R2 commands so they use persisted local storage (Add --local option to kv commands workers-sdk#3684 and Add --local to r2 object commands workers-sdk#3703)
• adding back getBindings(), getKVNamespace(), getR2Bucket(), getCaches(), getDurableObjectNamespace() ([Miniflare 3] ✨ Implement *magic* proxy and add back support for Miniflare#get*() methods #639)
• triggering scheduled events via CRON (Implement Fetcher#scheduled() for dispatching scheduled events workerd#961)
• step-through debugging (coming soon - 👀 Support for breakpoint debugging workerd#710)

[SupremeTechnopriest]

This looks great! Thanks for your hard work team. Looking forward to it!

[shaunco]

@admah - great to see these things being addressed in v3.

Any idea when getMiniflareFetchMock() will be linked back up to the work done in #632 ? It would seem that I could have my unit tests use the miniflare API directly to take advantage of #632 now, but I'm really struggling to get any of my tests working inside vitest with environment: miniflare removed and my own miniflare instance. If I go back to environment: miniflare my tests all execute great, but when my worker does fetch calls on external stuff fetchMock is not working...

edit: looks like my failure to use miniflare API directly was caused by setting scriptPath to typescript, which doesn't seems supported (even though it is supported when using environment: miniflare and can be wrangler deploy'd without emitting js).

edit2: turns out I'm just stuck in hell between wrangler2/miniflare2 and wrangler3/miniflare 3... I'll spend some more time on this before I rage quit, but this limbo state for unit tests is rough at the moment.