featrue: Include go-ds-pebble as built-in plugin #10367
Conversation
| @@ -23,6 +23,10 @@ The legacy subset of the Kubo RPC that was available via the Gateway port and wa | |||
|
|
|||
| If you have a legacy software that relies on this behavior, and want to expose parts of `/api/v0` next to `/ipfs`, use reverse-proxy in front of Kubo to mount both Gateway and RPC on the same port. NOTE: exposing RPC to the internet comes with security risk: make sure to specify access control via [API.Authorizations](https://github.com/ipfs/kubo/blob/master/docs/config.md#apiauthorizations). | |||
|
|
|||
| #### DataStore: Include pebble as built-in plugin | |||
|
|
|||
| support pebble in datadstore, same like leveldb. You can read more in <https://github.com/ipfs/kubo/issues/10347>. | |||
There was a problem hiding this comment.
| support pebble in datadstore, same like leveldb. You can read more in <https://github.com/ipfs/kubo/issues/10347>. | |
| Support pebble in datastore, same like leveldb. You can read more in <https://github.com/ipfs/kubo/issues/10347>. |
| ## pebbleds | ||
|
|
||
| Uses [pebble](https://github.com/cockroachdb/pebble) as a key value store | ||
|
|
||
| ```json | ||
| { | ||
| "type": "pebbleds", | ||
| "path": "<location of pebble inside repo>" | ||
| } | ||
| ``` |
There was a problem hiding this comment.
@hsanjuan I know pebble is used by default in IPFS Cluster, and there is a lot of custom configuration in https://github.com/ipfs-cluster/ipfs-cluster/blob/v1.0.8/datastore/pebble/config.go#L25-L70
Two questions:
- Does it make sense to expose them all via config here (as optional, use only when set)
- For implicit defaults, should we use defaults from pebble, or ones from ipfs-cluster?
There was a problem hiding this comment.
@lidel it makes sense to expose them, to be honest. Why not? I would expose them explicitally like cluster does. It is the only way to adjust things to usage, unlike the current badger-datastore defaults.
For implicit defaults, you need to look into what Kubo does. Kubo has its own block-cache, etc. so might as well disable/reduce pebble's cache, for example. Also need to adapt levels and granularity to the average 256K default block size.
|
I will do this in the next few days |
|
Triage question: @guojidan will you have interest and time to work on this, or is it ok for someone else to pick up the pebble work? |
I am so sorry. I have no time to do this work. If someone is interested in this, please feel free to take it. Thanks. |
|
@guojidan no problem! @gammazero will pick this up after we are done with more pressing stuff. closing this PR, we will continue in #10347 + new one. |
closed: #10347
I have tested it locally, It seems to be working properly 🤔