Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a multicodec.Registry type available. #172

Merged
merged 1 commit into from
May 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions linking/cid/linksystem.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,28 @@ import (
"github.com/ipld/go-ipld-prime/multicodec"
)

// DefaultLinkSystem returns an ipld.LinkSystem which uses cidlink.Link for ipld.Link.
// During selection of encoders, decoders, and hashers, it examines the multicodec indicator numbers and multihash indicator numbers from the CID,
// and uses the default global multicodec registry (see the go-ipld-prime/multicodec package) for resolving codec implementations,
// and the default global multihash registry (see the go-multihash/core package) for resolving multihash implementations.
//
// No storage functions are present in the returned LinkSystem.
// The caller can assign those themselves as desired.
func DefaultLinkSystem() ipld.LinkSystem {
return LinkSystemUsingMulticodecRegistry(multicodec.DefaultRegistry)
}

// LinkSystemUsingMulticodecRegistry is similar to DefaultLinkSystem, but accepts a multicodec.Registry as a parameter.
//
// This can help create a LinkSystem which uses different multicodec implementations than the global registry.
// (Sometimes this can be desired if you want some parts of a program to support a more limited suite of codecs than other parts of the program,
// or needed to use a different multicodec registry than the global one for synchronization purposes, or etc.)
func LinkSystemUsingMulticodecRegistry(mcReg multicodec.Registry) ipld.LinkSystem {
return ipld.LinkSystem{
EncoderChooser: func(lp ipld.LinkPrototype) (ipld.Encoder, error) {
switch lp2 := lp.(type) {
case LinkPrototype:
fn, err := multicodec.LookupEncoder(lp2.GetCodec())
fn, err := mcReg.LookupEncoder(lp2.GetCodec())
if err != nil {
return nil, err
}
Expand All @@ -28,7 +44,7 @@ func DefaultLinkSystem() ipld.LinkSystem {
lp := lnk.Prototype()
switch lp2 := lp.(type) {
case LinkPrototype:
fn, err := multicodec.LookupDecoder(lp2.GetCodec())
fn, err := mcReg.LookupDecoder(lp2.GetCodec())
if err != nil {
return nil, err
}
Expand Down
85 changes: 85 additions & 0 deletions multicodec/defaultRegistry.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
package multicodec

import (
"github.com/ipld/go-ipld-prime"
)

// DefaultRegistry is a multicodec.Registry instance which is global to the program,
// and is used as a default set of codecs.
//
// Some systems (for example, cidlink.DefaultLinkSystem) will use this default registry,
// which makes it easier to write programs that pass fewer explicit arguments around.
// However, these are *only* for default behaviors;
// variations of functions which allow explicit non-default options should always be available
// (for example, cidlink also has other LinkSystem constructor functions which accept an explicit multicodec.Registry,
// and the LookupEncoder and LookupDecoder functions in any LinkSystem can be replaced).
//
// Since this registry is global, mind that there are also some necessary tradeoffs and limitations:
// It can be difficult to control exactly what's present in this global registry
// (Libraries may register codecs in this registry as a side-effect of importing, so even transitive dependencies can affect its content!).
// Also, this registry is only considered safe to modify at package init time.
// If these are concerns for your program, you can create your own multicodec.Registry values,
// and eschew using the global default.
var DefaultRegistry = Registry{}

// RegisterEncoder updates the global DefaultRegistry to map a multicodec indicator number to the given ipld.Encoder function.
// The encoder functions registered can be subsequently looked up using LookupEncoder.
// It is a shortcut to the RegisterEncoder method on the global DefaultRegistry.
//
// Packages which implement an IPLD codec and have a multicodec number associated with them
// are encouraged to register themselves at package init time using this function.
// (Doing this at package init time ensures the default global registry is populated
// without causing race conditions for application code.)
//
// No effort is made to detect conflicting registrations in this map.
// If your dependency tree is such that this becomes a problem,
// there are two ways to address this:
// If RegisterEncoder is called with the same indicator code more than once, the last call wins.
// In practice, this means that if an application has a strong opinion about what implementation for a certain codec,
// then this can be done by making a Register call with that effect at init time in the application's main package.
// This should have the desired effect because the root of the import tree has its init time effect last.
// Alternatively, one can just avoid use of this registry entirely:
// do this by making a LinkSystem that uses a custom EncoderChooser function.
func RegisterEncoder(indicator uint64, encodeFunc ipld.Encoder) {
DefaultRegistry.RegisterEncoder(indicator, encodeFunc)
}

// LookupEncoder yields an ipld.Encoder function matching a multicodec indicator code number.
// It is a shortcut to the LookupEncoder method on the global DefaultRegistry.
//
// To be available from this lookup function, an encoder must have been registered
// for this indicator number by an earlier call to the RegisterEncoder function.
func LookupEncoder(indicator uint64) (ipld.Encoder, error) {
return DefaultRegistry.LookupEncoder(indicator)
}

// RegisterDecoder updates the global DefaultRegistry a map a multicodec indicator number to the given ipld.Decoder function.
// The decoder functions registered can be subsequently looked up using LookupDecoder.
// It is a shortcut to the RegisterDecoder method on the global DefaultRegistry.
//
// Packages which implement an IPLD codec and have a multicodec number associated with them
// are encouraged to register themselves in this map at package init time.
// (Doing this at package init time ensures the default global registry is populated
// without causing race conditions for application code.)
//
// No effort is made to detect conflicting registrations in this map.
// If your dependency tree is such that this becomes a problem,
// there are two ways to address this:
// If RegisterDecoder is called with the same indicator code more than once, the last call wins.
// In practice, this means that if an application has a strong opinion about what implementation for a certain codec,
// then this can be done by making a Register call with that effect at init time in the application's main package.
// This should have the desired effect because the root of the import tree has its init time effect last.
// Alternatively, one can just avoid use of this registry entirely:
// do this by making a LinkSystem that uses a custom DecoderChooser function.
func RegisterDecoder(indicator uint64, decodeFunc ipld.Decoder) {
DefaultRegistry.RegisterDecoder(indicator, decodeFunc)
}

// LookupDecoder yields an ipld.Decoder function matching a multicodec indicator code number.
// It is a shortcut to the LookupDecoder method on the global DefaultRegistry.
//
// To be available from this lookup function, an decoder must have been registered
// for this indicator number by an earlier call to the RegisterDecoder function.
func LookupDecoder(indicator uint64) (ipld.Decoder, error) {
return DefaultRegistry.LookupDecoder(indicator)
}
110 changes: 0 additions & 110 deletions multicodec/multicodec.go

This file was deleted.

83 changes: 83 additions & 0 deletions multicodec/registry.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package multicodec

import (
"fmt"

"github.com/ipld/go-ipld-prime"
)

// Registry is a structure for storing mappings of multicodec indicator numbers to ipld.Encoder and ipld.Decoder functions.
//
// The most typical usage of this structure is in combination with an ipld.LinkSystem.
// For example, a linksystem using CIDs and a custom multicodec registry can be constructed
// using cidlink.LinkSystemUsingMulticodecRegistry.
//
// Registry includes no mutexing. If using Registry in a concurrent context, you must handle synchronization yourself.
// (Typically, it is recommended to do initialization earlier in a program, before fanning out goroutines;
// this avoids the need for mutexing overhead.)
//
// go-ipld-prime also has a default registry, which has the same methods as this structure, but are at package scope.
// Some systems, like cidlink.DefaultLinkSystem, will use this default registry.
// However, this default registry is global to the entire program.
// This Registry type is for helping if you wish to make your own registry which does not share that global state.
//
// Multicodec indicator numbers are specified in
// https://github.com/multiformats/multicodec/blob/master/table.csv .
// You should not use indicator numbers which are not specified in that table
// (however, there is nothing in this implementation that will attempt to stop you, either; please behave).
type Registry struct {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we should specify the rules around when you can register encoders/decoders either here or in the LinkSystem constructor.

Similarly, should we be adding a mutex lock around the registry operations?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a glut of additional documentation about the synchronization (or rather, lack of it, and caller's responsibility).

encoders map[uint64]ipld.Encoder
decoders map[uint64]ipld.Decoder
}

func (r *Registry) ensureInit() {
if r.encoders != nil {
return
}
r.encoders = make(map[uint64]ipld.Encoder)
r.decoders = make(map[uint64]ipld.Decoder)
}

// RegisterEncoder updates a simple map of multicodec indicator number to ipld.Encoder function.
// The encoder functions registered can be subsequently looked up using LookupEncoder.
func (r *Registry) RegisterEncoder(indicator uint64, encodeFunc ipld.Encoder) {
r.ensureInit()
if encodeFunc == nil {
panic("not sensible to attempt to register a nil function")
}
r.encoders[indicator] = encodeFunc
}

// LookupEncoder yields an ipld.Encoder function matching a multicodec indicator code number.
//
// To be available from this lookup function, an encoder must have been registered
// for this indicator number by an earlier call to the RegisterEncoder function.
func (r *Registry) LookupEncoder(indicator uint64) (ipld.Encoder, error) {
encodeFunc, exists := r.encoders[indicator]
if !exists {
return nil, fmt.Errorf("no encoder registered for multicodec code %d (0x%x)", indicator, indicator)
}
return encodeFunc, nil
}

// RegisterDecoder updates a simple map of multicodec indicator number to ipld.Decoder function.
// The decoder functions registered can be subsequently looked up using LookupDecoder.
func (r *Registry) RegisterDecoder(indicator uint64, decodeFunc ipld.Decoder) {
r.ensureInit()
if decodeFunc == nil {
panic("not sensible to attempt to register a nil function")
}
r.decoders[indicator] = decodeFunc
}

// LookupDecoder yields an ipld.Decoder function matching a multicodec indicator code number.
//
// To be available from this lookup function, an decoder must have been registered
// for this indicator number by an earlier call to the RegisterDecoder function.
func (r *Registry) LookupDecoder(indicator uint64) (ipld.Decoder, error) {
decodeFunc, exists := r.decoders[indicator]
if !exists {
return nil, fmt.Errorf("no decoder registered for multicodec code %d (0x%x)", indicator, indicator)
}
return decodeFunc, nil
}