Separate package for trusted setups

[paulmillr]

While developing a kzg.ts package, it came to my understanding that including trusted setups would increase NPM pkg size massively.

Here's the idea:

1. I publish micro-trusted-setups or trusted-setups (or any other better name?)
2. The package will contain at least two files: default 400KB trusted setup and "fast" 800KB one
   • Difference between 400KB/800KB: initialization time (3 sec in js VS instant)
   • 800KB file would be auto-generated with script from 400KB
3. The setup files would be in typescript format, with proper typing. I think it's better than JSON
   • JS files are 30% smaller size than json, because of backtick strings
   • JSON files can't always be loaded properly without bundlers. For example, even in node.js json loaders are still experimental
4. We will be able to use the packages in both of our libraries.
   • Even better: maybe we do NOT depend on the setups by default
   • KZG usage is really rare on frontend-side

The files are already in eth-signer, but should be in a separate repo: small.ts, fast.ts

[jxom]

What if you snag the kzg NPM name (looks free), and publish the KZG implementation + trusted setups there?

import { KZG } from 'kzg'
import { setup } from 'kzg/setups/mainnet'

const kzg = new KZG(setup)

^ similar pattern to @scure/bip39

Update: I reserved the kzg NPM package (in case someone sees this comment and snipes it), but can transfer to you if you want to use it.

[paulmillr]

The pkg name is cool.

I’m becoming less of a fan of maintaining tens of packages. So ideally wanted to keep it inside of eth signer. Which is small itself and treeshaken.

For one example, signer contains special ssz impl, which is like 8x smaller than other one. At some point, people would want to use ssz.

What are your thoughts? How does viem currently include kzg / setups? How do you see that in the future?

[jxom]

We are just shipping the mainnet & minimal JSON trusted setup files right now: https://www.npmjs.com/package/viem?activeTab=code (under trusted-setups/) since the c-kzg binding doesn't support JS setup objects. Consumers import pointers to the files via viem/node entrypoint. For KZG interface, consumers currently have to plug in their own KZG implementation (c-kzg or kzg-wasm).

For the future, we wanted to rely on a purely isomorphic JS solution that works across browsers and runtimes (which I think I have mentioned to you in the past) – which seems like kzg.ts achieves that. Would be keen to include that code in Viem, and ideally entirely remove the defineKzg / setupKzg APIs so we can just import the isomorphic JS interface directly.

Is the implementation complete?

[paulmillr]

@jxom we can use kzg package to place this: https://github.com/paulmillr/trusted-setups

Or we can use @paulmillr/trusted-setups or micro-trusted-setups. Which one would you prefer?

I've ensured the library interops ok with c-kzg.

[jxom]

Apologies for the delayed response to your previous comment – was having a think about it.

Whatever package name sounds good – trust your intuition with that.

As for integration, I think it might be best if we still stick to the same inversion of control pattern we have now (instantiate kzg yourself – some people may still want to use c-kzg or kzg-wasm). We could modify our docs and guides to reference kzg setup via kzg.ts.

It would be cool if kzg.ts had complete API interop with c-kzg (ie. blobToKzgCommitment + computeBlobKzgProof were uint8array -> uint8array instead of string -> string).

edit: saw this ethereumjs/ethereumjs-monorepo#3662 (comment) – our Kzg interface is currently uint8array -> uint8array b/c of c-kzg, but could probably adapt to string -> string too.

[paulmillr]

@jxom I understand C uses Uint8Arrays. However, is it "right"?

I saw someone publishing first mainnet blob using viem. How exactly do you build blobs?

[paulmillr]

Some kind of u8a compat layer is easy to expose in kzg.ts, that's really simple.

[jxom]

I understand C uses Uint8Arrays. However, is it "right"?

Wouldn't say it's the most ideal case for our scenarios as string conversion will be needed anyway before sending over the wire (for blob transactions).

How exactly do you build blobs?

End up converting to strings before sending over the network (for blob transactions). Which is pretty much same process as you are doing now. However, consumers are still able to build Uint8Array blobs, and compute commitments & proofs for them in Uint8Array form (without any conversion) – though, not too sure if anyone is doing this.

Some kind of u8a compat layer is easy to expose in kzg.ts, that's really simple

Up to you. Either way, we can figure out something to make Viem work with string -> string interfaces too.

[paulmillr]

The package is now live on NPM at @paulmillr/trusted-setups.

[jxom]

Sweet! Will take a look.