refactor swing-store/makeSnapStore

[warner]

What is the Problem Being Solved?

We should refactor the various pieces of the kernel's state management library. The kernel currently needs three kinds of storage:

• the key-value store (currently provided by LMDB, code in swing-store-lmdb)
• the transcript stream store (currently provided by SQLite, previously by linear files, code in swing-store-lmdb)
• the XS snapshot store (currently provided by makeSnapStore, code in xsnap)

To build a snapshot store, the caller must provide a whole bunch of filesystem authorities, and merge the result with an object that comes back from the kernel's provideHostStorage() function, in a pattern that looks like this:

  const snapstorePath = path.resolve(
    __dirname,
    './fixture-test-reload-snapshot-gc/',
  );
  fs.mkdirSync(snapstorePath, { recursive: true });
  t.teardown(() => fs.rmdirSync(snapstorePath, { recursive: true }));

  const snapStore = makeSnapStore(snapstorePath, {
    tmpName,
    existsSync: fs.existsSync,
    createReadStream: fs.createReadStream,
    createWriteStream: fs.createWriteStream,
    rename: fs.promises.rename,
    unlink: fs.promises.unlink,
    resolve: path.resolve,
  });
  const hostStorage = { snapStore, ...provideHostStorage() };

This is repeated in both host applications (chain and solo), as well as several swingset unit tests.

I'd like to see this refactored to remove that code duplication.

I don't know how/if that squares with @dckc 's laudable goal of reducing the authority in the imported snapstore source code (i.e. continuing to avoid an import fs from 'fs' in that module, and requiring the caller to do that import instead).

Also, we need to rationalize the disparate locations of the source code for these three pieces, and make some decisions about how/if to continue supporting alternate DB tools. E.g. most swingset unit tests (and swingset-using tests in downstream consumer packages like Zoe/ERTP) are content with an in-RAM swingstore, which is easier to use because you don't have to pick a directory for it. The tests that need a snapstore must provide a directory, but those tests would be easier to write if it didn't have to pick one (e.g. keep the snapshots in RAM, and use tempfiles to have a real file on disk for the brief period when xsnap needs to read or write. Or make an automatically-deleting tempdir when the swingstore is created, and use it, instead of a named outlives-the-test directory).

cc @FUDCo @dckc

[warner]

Between the three of us, we might need to make this change now, rather than later, to support @dckc 's #3505 work. Or at least some subset of it, enough to allow the snapstore to be told to delete snapshots at the right time (just after the host has committed to the new swingset state, and before swingset has a chance to change any vat's state).

[dckc]

yes, I made a small step in 73573da

[dckc]

@warner I think maybe this is done as part of #3505 (317959d).

Or is there more to do?

[dckc]

swing-store-lmdb has become a misnomer and we should rename it, suggests @warner , so there is more to do.