Using Helia in Service Workers and Compatibility with Kubo

[orwithout]

I am exploring the possibility of running a Helia node within a Service Worker and potentially using Kubo on the local machine. I aim to use the same interface for both the Service Worker Helia node and the Kubo node without modifications, leveraging the RPC API provided by Kubo (Kubo RPC API Documentation).

Goal:

• To ensure seamless communication between the frontend, Service Worker-based Helia node, and a local Kubo node.

Could you provide insights or recommendations on achieving this interoperability? Are there known issues or considerations specific to running Helia in a Service Worker environment?

[SgtPooki]

FYI, we are running Helia in a service worker with https://github.com/ipfs-shipyard/service-worker-gateway. Older versions of the sw-gateway used a p2p Helia node, and newer versions might also enable p2p, but it currently only uses @helia/verified-fetch which uses @helia/http.

FYI, it should work, but you will likely need to manage the following issues:

1. service worker idle shutting down open streams/connections
   • How to listen to browser shutting down SW, and when it does, gracefully stopping the node.
   • Listening for starting back up (just 'activate' IIRC) and gracefully starting the node back up.
2. service worker new registration deactivating previously registered sw, i.e. the currently running Helia node (which may have active streams/conns)
3. persistent datastore/blockstore decisions
   • Do you want to persist data in the service worker?
     1. You should persist peerStore (datastore)
     2. You will likely need to manage capacity somehow for both datastore & blockstore (js-stores do not provide datastore/blockstores that limit the size of the store.. I opened feat: add tiered-limit datastore js-stores#301 to address this and will likely be publishing that as a package under @sgtpooki npm scope since it was requested to publish it separately instead of merging into the js-stores repo)

Also FYI: Firefox will likely provide the best environment since it has a functional webtransport implementation and Chromium doesn't yet (we have some folks working on this). Check out https://libp2p-webtransport-sessions.on.fleek.co/ in chromium and firefox to see just how much better connections are in Firefox.

[orwithout]

Thank you for your detailed response—it’s greatly appreciated and has bolstered my confidence in exploring the use of Helia. I’ve looked into the service-worker-gateway and @helia/verified-fetch projects. I plan to keep a close eye on these and will start using @helia/verified-fetch. Additionally, I intend to integrate OrbitDB to initialize a Helia node in the browser.

I have already made some initial implementations for running a persistent Helia node within a service worker, which can be seen here: senseurl - Helia in SW. Indeed, I am utilizing the 'activate' event as follows:

self.addEventListener('activate', event => {
    event.waitUntil(
        clients.claim().then(() => {
            console.log('Service Worker now controls the clients immediately.');
        })
    );
});

I am also following the js-stores pull request #301. Is there a reason this will not be merged into the main js-stores repo? This is a bit concerning for me, and I have opened an issue to discuss it further: Issue #526.

As for my persistent Helia node, the number of stored peers continues to increase, and I am still searching for effective ways to manage this growth. Any further advice or recommendations would be immensely helpful.

[SgtPooki]

AFAIK, you need a custom solution (lodestar has their own that runs in prod, and the PR I linked is an attempt at a generic solution that is not battle-tested) to manage the ever growing peer list. Also, I believe there are some changes coming down from libp2p that will reduce the size the peerstore takes up, but I may be mistaken there.

I think the main reasons it's not in the core repo is:

1. It's not battle-hardened yet. I believe the note from Alex was to deploy it where we want to test it and make sure it's working as expected, before merging into the js-stores repo
2. Also it's really hard to recommend a peer-limiting solution because limiting the peerStore size will directly impact how useful a node can be, and how useful the peers it finds can be.

[orwithout]

I understand the cautious approach towards merging js-stores pull request #301, and I've found the implementation in the Lodestar repository (here) to be a crucial reference. I plan to adapt similar strategies in my setup.

Thank you again for the valuable guidance. I am eager for any further suggestions or updates on this matter and am excited about integrating Helia (and OrbitDB) into my applications.