Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exchange signed routing records in identify #747

Merged
merged 46 commits into from
Apr 30, 2020

Conversation

yusefnapora
Copy link
Contributor

@yusefnapora yusefnapora commented Nov 21, 2019

This is still in progress & gomod is pointing at these PR branches:

TODO:

  • test
  • land dependency PRs and update gomod

@vyzo this should be good enough to start hacking on

@vyzo
Copy link
Contributor

vyzo commented Nov 21, 2019

thank you!

Copy link
Contributor

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good at first glance, just a couple of minor issues.

p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
@vyzo vyzo added topic/filecoin Topic filecoin P0 Critical: Tackled by core team ASAP labels Nov 21, 2019
@yusefnapora
Copy link
Contributor Author

@vyzo how do you feel about my last commit (028fe98)?

I added a config option to disable sending or parsing signed address records so that we could test the behavior where one peer supports them but the other doesn't. It kind of bugs me though, since the flag is only really useful for testing and it clutters up the code. I couldn't figure out a better way to test it though, short of spinning up both versions for real in something like testground.

@vyzo
Copy link
Contributor

vyzo commented Nov 27, 2019

Hrm, it's not ideal but we do need to test somehow.
Perhaps give it a more explicit name that indicates it's only used for testing?

@yusefnapora
Copy link
Contributor Author

@vyzo I updated this branch based on @raulk's design blueprint. It points to a new commit from the -core PR that defines some new event types, and I rebased this branch and the -core PR onto master, since there were a few dependency updates since I opened the PRs.

The gist is that now BasicHost emits a new EvtLocalAddressesUpdated event when it detects addresses have changed, instead of directly calling ids.Push().

There's also a new routingStateManager component in the identify service that listens for EvtLocalAddressesUpdated events and builds a new SignedRoutingState record. It emits a LocalPeerRoutingStateUpdated event, which the main identify service listens for. When the id service gets that event, it triggers the identify/push.

This adds a bit more indirection, but it should give you a way to grab new routing state record in other places outside the ID service (e.g. peer exchange).

Unfortunately, the mock network tests seem to have gotten flakier - I had to re-run them on Travis, and they seem to fail pretty often on my laptop. I'm not sure if I actually made it worse, or if there's just more spam in the error log - there's a ton of error messages from boguskey ("this better be a test!"), since it's using bogus testing keys to sign/verify the routing records.

Since the mock net tests have some assertions about timing (e.g. "expected to take ~2 seconds, took x"), it's possible that I made performance regress enough to fail the tests. It's odd that they'd fail with the bogus keys though, since they don't actually do any expensive crypto. But perhaps the event bus messes with the timing, or just the overhead of marshaling / unmarshaling the records? I've had these tests fail before, so maybe it's unrelated, but they definitely spew out a lot more errors now, thanks to the bogus key thing.

Anyway, hopefully this helps with the peer exchange stuff you're working on - you should be able to listen for EvtLocalRoutingStateUpdated from anywhere and get the latest records.

I'm off next week, but you can ping me if you're blocked on this. I'm going to switch this PR to "ready for review", but we'll need to merge the -core and -peerstore PRs first and update gomod here before we can merge this one, since it's pinned to specific commits from those branches at the moment.

@yusefnapora yusefnapora changed the title wip - exchange signed routing records in identify exchange signed routing records in identify Dec 20, 2019
@yusefnapora yusefnapora marked this pull request as ready for review December 20, 2019 22:02
@vyzo
Copy link
Contributor

vyzo commented Dec 21, 2019

Is Push still called somehow? We need this to propagate our address changes to our peers.

@yusefnapora yusefnapora mentioned this pull request Jan 23, 2020
10 tasks
@yusefnapora
Copy link
Contributor Author

@raulk, @vyzo, etc

I realized that I may be aiming a gun at our foot with respect to LAN-local addresses.

As it is now in the PR, the default behavior is to filter LAN and loopback addresses out of PeerRecords before we send them out in identify, etc. There's a flag that can change this behavior so that you can include local addresses, but at the moment it's only on the IDService; I was about to bubble the option up to the libp2p.New options when I stopped to think a moment.

There are two behaviors that are problematic in tandem:

  • The Peerstore will replace any existing addrs for a peer with the addrs from a PeerRecord
  • There's currently nothing stopping us from sending out empty PeerRecords. A peer with only local addrs will still send out a record with an empty Addrs list after filtering the local addrs out. The receiver will then drop the existing local addrs it knows for the peer and replace with the empty set from the PeerRecord.

This effectively means that if you don't set the option to include LAN addresses in PeerRecords, anyone that happens to find your LAN address out-of-band will forget it as soon as they talk to you :(

I'd rather not make interacting with LAN peers worse by default, which I think this does. I still need to write a test case to confirm that this is happening, btw. But I started to suspect that this was a problem based on some odd behavior in earlier tests, so I don't think I'm chasing a ghost.

I'm going to write that test, but in the meantime if anyone has ideas for addressing this let me know.

The simplest option that I can think of is to change the default, so that you have to opt-out of including local addrs in PeerRecords. That has us putting a lot of irrelevant addresses on the public DHT though...

Alternatively, we could prevent you from sending signed PeerRecords in identify with no addrs in them, and change the peerstore so that it ignores records with no addrs instead of removing existing addrs. That should force everyone to fallback to the unsigned addrs that are still being sent in identify alongside the peer record, and which aren't filtered.

A more involved and annoying solution would be to have multiple kinds of PeerRecord, one with local addrs and one for public sharing on the DHT. But we'd have to revisit some assumptions to make that work.

Other ideas?

@yusefnapora
Copy link
Contributor Author

hmm, looks like I may have introduced a race condition, and/or my tests are buggy. will investigate :)

@yusefnapora
Copy link
Contributor Author

Regarding the problem with LAN addresses above, @raulk and I discussed in a sync chat just now and agreed that changing the default (so that LAN addresses are included in peer records) is the best option for now. This essentially gives us the status quo, but with signatures, since we're exchanging LAN addrs in identify now.

In the future, we can add specialized peer records with different scope (e.g. LAN-only, public DHT, etc), so that we can keep LAN addrs out of the records that get published globally.

Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main request is to version the identify and identify push protocols.

p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/host/basic/basic_host_test.go Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM barring the usage of a pointer in the EvtLocalPeerRecordUpdated event.

Already spoke to @yusefnapora and he's going to fix that upstream, then update this PR. Approving in anticipation!

Excellent work!

p2p/host/basic/basic_host_test.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
@raulk
Copy link
Member

raulk commented Feb 10, 2020

Reminder: we need to make a go-libp2p-core release and update the go.mod here, before we merge this. Let's:

  • Fix the EvtLocalPeerRecordUpdated event.
  • Obtain sign-off from @aschmahmann and/or @Stebalien that all this work, in the current shape and form, is indeed easily integratable into go-libp2p-kad-dht.

willscott added a commit to willscott/go-libp2p-autonat that referenced this pull request Feb 14, 2020
* incoming connections post a channel event - fix libp2p#40
* inbound connections reduce the frequency of probes - address libp2p#35

waiting on libp2p/go-libp2p#747 for detecting local address changes
willscott added a commit to willscott/go-libp2p-autonat that referenced this pull request Feb 14, 2020
* incoming connections post a channel event - fix libp2p#40
* inbound connections reduce the frequency of probes - address libp2p#35

waiting on libp2p/go-libp2p#747 for detecting local address changes
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/host/basic/basic_host.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
p2p/protocol/identify/peer_record.go Outdated Show resolved Hide resolved
willscott added a commit to willscott/go-libp2p-autonat that referenced this pull request Feb 17, 2020
* incoming connections post a channel event - fix libp2p#40
* inbound connections reduce the frequency of probes - address libp2p#35

waiting on libp2p/go-libp2p#747 for detecting local address changes
@aarshkshah1992
Copy link
Contributor

ping @yusefnapora @raulk .

Copy link
Contributor Author

@yusefnapora yusefnapora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aarshkshah1992 thanks again for taking this over the line - it won't let me "approve" since I opened the PR, but I like this branch much better now!

This looks good to merge, pending @raulk's review :)

Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments. Will do a second and final pass tomorrow.

Comment on lines 326 to 327
case h.addrChangeChan <- struct{}{}:
default:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine because the channel is buffered. If the event loop is firing, we will queue the next trigger. If the channel is full, the next iteration of the event loop will pick up the update we were intending to communicate.


emitAddrChange := func(currentAddrs []ma.Multiaddr) {
changeEvt := makeUpdatedAddrEvent(lastAddrs, currentAddrs)
if changeEvt != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Negate the condition to short-circuit instead, and outdent the block.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 411 to 415
if h.Addrs() != nil {
emitAddrChange(h.Addrs())
}
// init lastAddrs
lastAddrs = h.Addrs()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is racy. Capture the addresses and use them when emitting the event and setting lastAddrs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@raulk Even though this code is executed in a single dedicated go-routine, have refactored this to make it easier to reason about.

Comment on lines 431 to 433
emitAddrChange(h.Addrs())
// update last seen addrs
lastAddrs = h.Addrs()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Racy. Capture, emit and set.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 13 to 14
"github.com/libp2p/go-libp2p-core/peerstore"
"github.com/libp2p/go-libp2p-core/record"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import order.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 21 to 24
"github.com/libp2p/go-libp2p-core/peer"
"github.com/libp2p/go-libp2p-core/protocol"
"github.com/libp2p/go-libp2p-core/test"
"github.com/libp2p/go-libp2p/p2p/protocol/identify"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import order.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

go.mod Outdated
github.com/jbenet/go-cienv v0.1.0
github.com/jbenet/goprocess v0.1.4
github.com/libp2p/go-conn-security-multistream v0.1.0
github.com/libp2p/go-eventbus v0.1.0
github.com/libp2p/go-eventbus v0.1.1-0.20200326052355-c30c9409b9a4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this bump?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we do not. This was the remanent of a previous eventbus change I made that we do not need anymore.

This has been fixed.

}

rec := peer.NewPeerRecord()
rec.PeerID = h.network.LocalPeer()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

host.ID()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
return next
case <-time.After(5 * time.Second):
t.Fatal("event not received in 5 seconds")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fatal must be called from the main goroutine: https://golang.org/pkg/testing/#T

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is being executed on the main goroutine, not on a separate one.

func (sma sortedMultiaddrs) Len() int { return len(sma) }
func (sma sortedMultiaddrs) Swap(i, j int) { sma[i], sma[j] = sma[j], sma[i] }
func (sma sortedMultiaddrs) Less(i, j int) bool {
return bytes.Compare(sma[i].Bytes(), sma[j].Bytes()) == 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn’t this be == -1?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely got rid of this code as we don't really need it.

@aarshkshah1992
Copy link
Contributor

@Stebalien Please can I get a review of this so we can merge it ?

@@ -142,10 +148,21 @@ func NewHost(ctx context.Context, net network.Network, opts *HostOpts) (*BasicHo
if h.emitters.evtLocalProtocolsUpdated, err = h.eventbus.Emitter(&event.EvtLocalProtocolsUpdated{}); err != nil {
return nil, err
}
if h.emitters.evtLocalAddrsUpdated, err = h.eventbus.Emitter(&event.EvtLocalAddressesUpdated{}); err != nil {
if h.emitters.evtLocalAddrsUpdated, err = h.eventbus.Emitter(&event.EvtLocalAddressesUpdated{}, eventbus.Stateful); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this should be stateful. Are we doing this to get around the fact that we don't have some form of GetPeerRecord() method as described in #747 (comment)? If so, we should just add that method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien We have a GetPeerRecord(p peer.ID) on the peerstore now.

However, calling that when you get a EvtLocalAddressChanged event could be racy as the address you see on the event could be different fro the signed record you get from the peerstore.

This avoids that race.

p2p/host/basic/basic_host.go Outdated Show resolved Hide resolved
p2p/host/basic/basic_host.go Outdated Show resolved Hide resolved
p2p/host/basic/basic_host.go Outdated Show resolved Hide resolved
// set listen addrs, get our latest addrs from Host.
laddrs := ids.Host.Addrs()
// Note: LocalMultiaddr is sometimes 0.0.0.0
viaLoopback := manet.IsIPLoopback(c.LocalMultiaddr()) || manet.IsIPLoopback(c.RemoteMultiaddr())
Copy link
Contributor

@aarshkshah1992 aarshkshah1992 Apr 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien

Hmm.. we probably want to imitate this logic for the signed addresses ? Currently, we put all our addresses on the signed record which dosen't sound right.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's probably also worth filtering manet.IsIPUnspecified() records in both cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we'd have multiple records with different scopes. But we can tackle that later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien Could you point me to an issue/discussion/PR to better understand the idea and scope of the address scoping work ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aarshkshah1992 I would imagine the intention would be to split record/addresses along the same policy distinction as in https://github.com/libp2p/go-libp2p-kad-dht/blob/master/dht_filters.go - a public scope for addresses the peer thinks might be valid for remote peers and is willing to drop on the public DHT, and separately the set of addresses valid for LAN peers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aarshkshah1992
Copy link
Contributor

ping @Stebalien .

@aarshkshah1992
Copy link
Contributor

aarshkshah1992 commented Apr 23, 2020

@Stebalien

I'm not sure if this should be stateful. Are we doing this to get around the fact that we don't have some form of GetPeerRecord() method as described in #747 (comment)? If so, we should just add that method.

Based on the new design we discussed, I think we can add /use this method because we plan to ignore the contents on the event for Identify now.

However, I'd like to merge this as is and do that change in the next PR if that's okay with you. Please can you give this one last review ?

Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My one concern is that we might want to change the protocol slightly when we add address scopes. I'm wondering if we should put this protocol behind a simple ExperimentSignedIdentify global variable flag so we can merge this and start testing it without accidentally deploying it on the network before we're ready.

if ok && signedPeerRecord != nil {
_, addErr := cab.ConsumePeerRecord(signedPeerRecord, ttl)
if addErr != nil {
log.Errorf("error adding signed addrs to peerstore: %v", addErr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is, could we get this error because someone sent us a bad record? If so, we shouldn't log an error.

p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Outdated Show resolved Hide resolved
p2p/protocol/identify/id.go Show resolved Hide resolved
// set listen addrs, get our latest addrs from Host.
laddrs := ids.Host.Addrs()
// Note: LocalMultiaddr is sometimes 0.0.0.0
viaLoopback := manet.IsIPLoopback(c.LocalMultiaddr()) || manet.IsIPLoopback(c.RemoteMultiaddr())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we'd have multiple records with different scopes. But we can tackle that later.

// in a form that lets us share authenticated addrs with other peers.
// see github.com/libp2p/go-libp2p-core/record/pb/envelope.proto and
// github.com/libp2p/go-libp2p-core/peer/pb/peer_record.proto for message definitions.
optional bytes signedPeerRecord = 8;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this repeated? That way, we can have one record per scope (and we can add scope info to the signed peer record itself).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: we can also do this in a new protocol version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien

If we do this in a new protocol version, will we make a "breaking" protocol change for it or add a new field for the multiple scoped records ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd make a breaking protocol change.

@aarshkshah1992
Copy link
Contributor

@Stebalien

I've addressed your comments. Just have some questions to better understand the address scoping work and the protocol changes we will need though they probably don't block this PR.

My one concern is that we might want to change the protocol slightly when we add address scopes. I'm wondering if we should put this protocol behind a simple ExperimentSignedIdentify global variable flag so we can merge this and start testing it without accidentally deploying it on the network before we're ready.

I'm not sure I understand what you mean. Do we want to block on the address scoping work before doing a full fledged release ?

@Stebalien
Copy link
Member

I'm not sure I understand what you mean. Do we want to block on the address scoping work before doing a full fledged release ?

Eh, not really. I guess I just wanted a way to play around with this first before we made it the default.

But honestly, it's fine. If we find we don't like something, we can always introduce a new protocol version.


// LegacyID is the protocol.ID of version 1.0.0 of the identify
// service, which does not support signed peer records.
const LegacyID = "/ipfs/id/1.0.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming:

ID1_1_0
ID1_0_0

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we're going to add more in the future.

@@ -85,12 +89,14 @@ type IDService struct {

addrMu sync.Mutex

peerrec *record.Envelope
peerrecMu sync.RWMutex
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: mutex-hat (lock goes above the record)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien We will anyways get rid of this once we have the GetRecord function in the next PR.

// set listen addrs, get our latest addrs from Host.
laddrs := ids.Host.Addrs()
// Note: LocalMultiaddr is sometimes 0.0.0.0
viaLoopback := manet.IsIPLoopback(c.LocalMultiaddr()) || manet.IsIPLoopback(c.RemoteMultiaddr())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Stebalien
Copy link
Member

Thank you for your patience! LGTM!

@aarshkshah1992 aarshkshah1992 merged commit 077a818 into master Apr 30, 2020
@aarshkshah1992 aarshkshah1992 deleted the feat/identify-signed-addrs branch April 30, 2020 05:36
@raulk
Copy link
Member

raulk commented May 12, 2020

This does not compile, and master is broken:

⟩ go build ./...
go: downloading github.com/libp2p/go-libp2p-core v0.5.2
go: downloading github.com/miekg/dns v1.1.28
# github.com/libp2p/go-libp2p/p2p/protocol/identify
p2p/protocol/identify/id.go:207:14: cannot use &rec (type **record.Envelope) as type *record.Envelope in assignment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P0 Critical: Tackled by core team ASAP topic/filecoin Topic filecoin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants