Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New net interface #69

Merged
merged 221 commits into from
Sep 22, 2014
Merged

New net interface #69

merged 221 commits into from
Sep 22, 2014

Conversation

jbenet
Copy link
Member

@jbenet jbenet commented Sep 14, 2014

No description provided.

@jbenet jbenet added the status/in-progress In progress label Sep 14, 2014
@jbenet
Copy link
Member Author

jbenet commented Sep 14, 2014

warning: not yet wired.

@btc btc mentioned this pull request Sep 14, 2014
@jbenet
Copy link
Member Author

jbenet commented Sep 14, 2014

#75 #74

return false
}
return bool(r[0]&0x80 == 0x80)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perfmode @whyrusleeping heads up -- Request uses the per-msg RequestID nonce to also identify whether it's a request or a response (based on the first bit). (lmk if you think this is too sketchy)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When can r == nil?

My concern would be around future-proofing. Will this representation be resistant to errors introduced by updates and versioning incompatibilities?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehhh... im going to lean towards 'sketchy'. Like, yeah it works, but it should be commented to hell and back if were going to keep it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@perfmode r == nil when messages do not want a RequestID associated -- i.e. they're not requests.

@whyrusleeping Yeah, it's sketchy. It feels so wrong to use a whole protobuf for a bit thought >.< -- i guess it is technically TRTTD.

// speaking to (i.e. peer.ID). In terms of moving around simple addresses
// -- instead of an (ID, Addr) pair -- we can use:
//
// /ip4/10.20.30.40/tcp/1234/ipfs/Qxhxxchxzcncxnzcnxzcxzm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have some sort of extra functionality added to multiaddr? Maybe add pluggable protocols so ipfs can specify the 'ipfs peer' protocol which denotes an id that is expeced from the connection?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, sounds good. we can allocate a multiaddr number and all.

think b58 is good enough to have multiaddr default to for protocols whose string representations it doesn't know? could do hex. Oh damnit b58 will run into trouble on case sensitive OSes....


// add all the peers we got first.
for _, p := range peers {
r.addPeerToQuery(p, nil) // don't have access to self here...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm... having a reference to outself here might be useful. I had issues without it before.

@btc
Copy link
Contributor

btc commented Sep 19, 2014

@jbenet Recall services register handlers on the network. These handlers are called when inbound messages arrive.

 HandleMessage(context.Context, msg.NetMessage) (msg.NetMessage, error)

What does it mean for the handler to return an error to the network layer?

@jbenet
Copy link
Member Author

jbenet commented Sep 19, 2014

@perfmode good point, that doesn't make much sense. See https://github.com/jbenet/go-ipfs/blob/net/net/service/service.go#L183-L188 Think it should just return a message, and handle/log/whatever its own error?

@btc
Copy link
Contributor

btc commented Sep 20, 2014

@jbenet @whyrusleeping merged this into net already, but still want to make sure you're aware.

#91

Brian Tiger Chow added 14 commits September 22, 2014 04:06
changing the bitswap interace breaks tests and makes things a bit
difficult going forward. I think I have a temporary solution to replace
the async method.

this commit partially reverts changes from:

ec50703

ec50703
until construction is complete on the actual async method

reverts changes from ec50703

ec50703
fix(bitswap) pass go vet

fixes #97

#97
true -> always send to peer

false -> use ledger-based strategy described in IPFS paper draft 3
failing due to nil pointer dereference
@btc
Copy link
Contributor

btc commented Sep 22, 2014

@jbenet @whyrusleeping

:shipit:

jbenet added a commit that referenced this pull request Sep 22, 2014
@jbenet jbenet merged commit 676d9c5 into master Sep 22, 2014
@jbenet jbenet removed the status/in-progress In progress label Sep 22, 2014
@jbenet
Copy link
Member Author

jbenet commented Sep 22, 2014

@btc
Copy link
Contributor

btc commented Sep 22, 2014

AHHHHHHHHHHHHHHH!!!!

@btc
Copy link
Contributor

btc commented Sep 22, 2014

The smile on my face right now.

@jbenet
Copy link
Member Author

jbenet commented Sep 22, 2014

Warning: remote port forwarding failed for listen port 52698
From github.com:jbenet/go-ipfs
 * branch            master     -> FETCH_HEAD
 Dockerfile                                                                 |  19 ++++
 Godeps/Godeps.json                                                         |  15 ++-
 Godeps/_workspace/src/code.google.com/p/go.net/context/context.go          | 431 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/code.google.com/p/go.net/context/context_test.go     | 553 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/code.google.com/p/go.net/context/withtimeout_test.go |  26 +++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/codec.go              |  28 +++--
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/index.go              |  21 +++-
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/multiaddr_test.go     |  38 ++++++-
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net.go                |  77 +++++++++++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net_test.go           |  49 +++++++++
 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/protocols.go          |   5 +-
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/README.md              |  30 ++++++
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub.go              | 208 ++++++++++++++++++++++++++++++++++++
 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub_test.go         | 230 +++++++++++++++++++++++++++++++++++++++
 README.md                                                                  |  10 +-
 bitswap/bitswap.go                                                         | 282 ------------------------------------------------
 bitswap/ledger.go                                                          |  51 ---------
 blocks/blocks.go                                                           |   3 +-
 blocks/blocks_test.go                                                      |  25 +++++
 blockservice/blockservice.go                                               |  21 ++--
 blockstore/blockstore.go                                                   |  47 ++++++++
 blockstore/blockstore_test.go                                              |  55 ++++++++++
 cmd/ipfs/add.go                                                            | 100 +++--------------
 cmd/ipfs/cat.go                                                            |  29 ++---
 cmd/ipfs/init.go                                                           |  60 +++++++++--
 cmd/ipfs/ipfs.go                                                           |  28 ++++-
 cmd/ipfs/ls.go                                                             |  24 +++--
 cmd/ipfs/mount_unix.go                                                     |  28 ++++-
 cmd/ipfs/refs.go                                                           |  58 +++-------
 config/config.go                                                           |  18 ++--
 config/config_test.go                                                      |  17 ++-
 config/serialize.go                                                        |   5 +
 {cmd/ipfs => core/commands}/add.go                                         |  77 +++++--------
 core/commands/cat.go                                                       |  29 +++++
 core/commands/ls.go                                                        |  22 ++++
 core/commands/pin.go                                                       |  23 ++++
 core/commands/refs.go                                                      |  65 +++++++++++
 core/core.go                                                               | 142 +++++++++++++++++-------
 core/core_test.go                                                          |  28 ++++-
 core/datastore.go                                                          |   7 +-
 crypto/key_test.go                                                         |  54 ++++++++++
 crypto/spipe/Makefile                                                      |   8 ++
 identify/identify.go => crypto/spipe/handshake.go                          | 166 +++++++++++++++++-----------
 {identify => crypto/spipe}/message.pb.go                                   |  46 ++++----
 crypto/spipe/message.proto                                                 |  14 +++
 crypto/spipe/pipe.go                                                       |  76 +++++++++++++
 daemon/daemon.go                                                           | 102 ++++++++++++++++++
 daemon/daemon_client.go                                                    |  29 +++++
 daemon/daemon_test.go                                                      |  71 ++++++++++++
 dev.md                                                                     |   9 +-
 doc.go                                                                     |   2 +
 exchange/bitswap/bitswap.go                                                | 185 ++++++++++++++++++++++++++++++++
 exchange/bitswap/bitswap_test.go                                           | 301 +++++++++++++++++++++++++++++++++++++++++++++++++++
 exchange/bitswap/message/Makefile                                          |   8 ++
 exchange/bitswap/message/message.go                                        |  93 ++++++++++++++++
 {bitswap => exchange/bitswap/message}/message.pb.go                        |   2 +-
 {bitswap => exchange/bitswap/message}/message.proto                        |   2 +-
 exchange/bitswap/message/message_test.go                                   | 172 +++++++++++++++++++++++++++++
 exchange/bitswap/network/interface.go                                      |  53 +++++++++
 exchange/bitswap/network/net_message_adapter.go                            |  93 ++++++++++++++++
 exchange/bitswap/notifications/notifications.go                            |  55 ++++++++++
 exchange/bitswap/notifications/notifications_test.go                       |  58 ++++++++++
 exchange/bitswap/strategy/interface.go                                     |  32 ++++++
 exchange/bitswap/strategy/ledger.go                                        |  93 ++++++++++++++++
 exchange/bitswap/strategy/ledger_test.go                                   |  23 ++++
 bitswap/strategy.go => exchange/bitswap/strategy/math.go                   |  11 +-
 bitswap/strategy_test.go => exchange/bitswap/strategy/math_test.go         |   2 +-
 exchange/bitswap/strategy/strategy.go                                      | 114 ++++++++++++++++++++
 exchange/bitswap/strategy/strategy_test.go                                 | 103 ++++++++++++++++++
 exchange/bitswap/testnet/network.go                                        | 174 ++++++++++++++++++++++++++++++
 exchange/bitswap/testnet/network_test.go                                   | 138 ++++++++++++++++++++++++
 exchange/bitswap/testnet/routing.go                                        |  97 +++++++++++++++++
 exchange/bitswap/testnet/routing_test.go                                   | 157 +++++++++++++++++++++++++++
 exchange/interface.go                                                      |  20 ++++
 exchange/offline/offline.go                                                |  34 ++++++
 exchange/offline/offline_test.go                                           |  28 +++++
 identify/identify_test.go                                                  |  54 ----------
 identify/message.proto                                                     |  14 ---
 importer/importer.go                                                       |   7 +-
 importer/importer_test.go                                                  |  74 ++++++++++---
 importer/rabin.go                                                          | 134 +++++++++++++++++++++++
 importer/split_test.go                                                     |  48 ---------
 importer/splitting.go                                                      |  91 ++++------------
 merkledag/dagreader.go                                                     |  13 ++-
 merkledag/merkledag.go                                                     |   4 +-
 {swarm => net/conn}/conn.go                                                |  62 +++++++----
 {swarm => net/conn}/conn_test.go                                           |  25 ++---
 net/interface.go                                                           |  49 +++++++++
 net/message/message.go                                                     |  78 ++++++++++++++
 net/mux/Makefile                                                           |   8 ++
 net/mux/mux.go                                                             | 188 ++++++++++++++++++++++++++++++++
 net/mux/mux.pb.go                                                          |  90 ++++++++++++++++
 net/mux/mux.proto                                                          |  13 +++
 net/mux/mux_test.go                                                        | 276 +++++++++++++++++++++++++++++++++++++++++++++++
 net/net.go                                                                 | 109 +++++++++++++++++++
 net/net_test.go                                                            |   1 +
 net/service/Makefile                                                       |   8 ++
 net/service/request.go                                                     | 127 ++++++++++++++++++++++
 net/service/request.pb.go                                                  |  50 +++++++++
 net/service/request.proto                                                  |   6 ++
 net/service/request_test.go                                                |  41 +++++++
 net/service/service.go                                                     | 224 ++++++++++++++++++++++++++++++++++++++
 net/service/service_test.go                                                | 146 +++++++++++++++++++++++++
 net/swarm/conn.go                                                          | 210 ++++++++++++++++++++++++++++++++++++
 net/swarm/swarm.go                                                         | 190 +++++++++++++++++++++++++++++++++
 {swarm => net/swarm}/swarm_test.go                                         |  83 ++++++++------
 peer/peer.go                                                               |  35 ++++--
 peer/peer_test.go                                                          |   8 +-
 peer/peerstore.go                                                          |  87 +++++++++++++++
 peer/peerstore_test.go                                                     |  82 ++++++++++++++
 peer/queue/distance.go                                                     | 101 ++++++++++++++++++
 peer/queue/interface.go                                                    |  18 ++++
 peer/queue/queue_test.go                                                   | 131 +++++++++++++++++++++++
 peer/queue/sync.go                                                         |  74 +++++++++++++
 routing/dht/Makefile                                                       |  11 ++
 routing/dht/Message.go                                                     |  58 +++++-----
 routing/dht/dht.go                                                         | 712 +++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------------------------------------------
 routing/dht/dht_logger.go                                                  |   5 +
 routing/dht/dht_test.go                                                    | 269 ++++++++++++++++++++++++++++------------------
 routing/dht/ext_test.go                                                    | 263 +++++++++++++++++++++++++--------------------
 routing/dht/handlers.go                                                    | 236 ++++++++++++++++++++++++++++++++++++++++
 routing/dht/messages.pb.go                                                 | 138 ++++++++++++------------
 routing/dht/messages.proto                                                 |  28 +++--
 routing/dht/query.go                                                       | 237 ++++++++++++++++++++++++++++++++++++++++
 routing/dht/routing.go                                                     | 422 ++++++++++++++++++++++++++++--------------------------------------------
 routing/kbucket/bucket.go                                                  |   2 +-
 routing/kbucket/table.go                                                   |   4 +-
 routing/kbucket/table_test.go                                              |   4 +-
 routing/kbucket/util.go                                                    |  54 ++--------
 routing/keyspace/keyspace.go                                               |  97 +++++++++++++++++
 routing/keyspace/xor.go                                                    |  74 +++++++++++++
 routing/keyspace/xor_test.go                                               | 147 +++++++++++++++++++++++++
 routing/routing.go                                                         |  13 +--
 swarm/interface.go                                                         |  20 ----
 swarm/mes_listener.go                                                      | 123 ---------------------
 swarm/mes_listener_test.go                                                 |  32 ------
 swarm/mes_wrapper.pb.go                                                    |  85 ---------------
 swarm/mes_wrapper.proto                                                    |  12 ---
 swarm/swarm.go                                                             | 506 --------------------------------------------------------------------------------------
 swarm/wrapper.go                                                           |  24 -----
 util/key_set.go                                                            |  46 ++++++++
 util/testutil/blocks.go                                                    |  22 ++++
 util/todocounter/counter.go                                                | 118 ++++++++++++++++++++
 util/util.go                                                               |  15 +++
 144 files changed, 9429 insertions(+), 2916 deletions(-)
 create mode 100644 Dockerfile
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/context.go
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/context_test.go
 create mode 100644 Godeps/_workspace/src/code.google.com/p/go.net/context/withtimeout_test.go
 create mode 100644 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net.go
 create mode 100644 Godeps/_workspace/src/github.com/jbenet/go-multiaddr/net_test.go
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/README.md
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub.go
 create mode 100644 Godeps/_workspace/src/github.com/tuxychandru/pubsub/pubsub_test.go
 delete mode 100644 bitswap/bitswap.go
 delete mode 100644 bitswap/ledger.go
 create mode 100644 blocks/blocks_test.go
 create mode 100644 blockstore/blockstore.go
 create mode 100644 blockstore/blockstore_test.go
 copy {cmd/ipfs => core/commands}/add.go (52%)
 create mode 100644 core/commands/cat.go
 create mode 100644 core/commands/ls.go
 create mode 100644 core/commands/pin.go
 create mode 100644 core/commands/refs.go
 create mode 100644 crypto/key_test.go
 create mode 100644 crypto/spipe/Makefile
 rename identify/identify.go => crypto/spipe/handshake.go (54%)
 rename {identify => crypto/spipe}/message.pb.go (52%)
 create mode 100644 crypto/spipe/message.proto
 create mode 100644 crypto/spipe/pipe.go
 create mode 100644 daemon/daemon.go
 create mode 100644 daemon/daemon_client.go
 create mode 100644 daemon/daemon_test.go
 create mode 100644 doc.go
 create mode 100644 exchange/bitswap/bitswap.go
 create mode 100644 exchange/bitswap/bitswap_test.go
 create mode 100644 exchange/bitswap/message/Makefile
 create mode 100644 exchange/bitswap/message/message.go
 rename {bitswap => exchange/bitswap/message}/message.pb.go (98%)
 rename {bitswap => exchange/bitswap/message}/message.proto (82%)
 create mode 100644 exchange/bitswap/message/message_test.go
 create mode 100644 exchange/bitswap/network/interface.go
 create mode 100644 exchange/bitswap/network/net_message_adapter.go
 create mode 100644 exchange/bitswap/notifications/notifications.go
 create mode 100644 exchange/bitswap/notifications/notifications_test.go
 create mode 100644 exchange/bitswap/strategy/interface.go
 create mode 100644 exchange/bitswap/strategy/ledger.go
 create mode 100644 exchange/bitswap/strategy/ledger_test.go
 rename bitswap/strategy.go => exchange/bitswap/strategy/math.go (55%)
 rename bitswap/strategy_test.go => exchange/bitswap/strategy/math_test.go (95%)
 create mode 100644 exchange/bitswap/strategy/strategy.go
 create mode 100644 exchange/bitswap/strategy/strategy_test.go
 create mode 100644 exchange/bitswap/testnet/network.go
 create mode 100644 exchange/bitswap/testnet/network_test.go
 create mode 100644 exchange/bitswap/testnet/routing.go
 create mode 100644 exchange/bitswap/testnet/routing_test.go
 create mode 100644 exchange/interface.go
 create mode 100644 exchange/offline/offline.go
 create mode 100644 exchange/offline/offline_test.go
 delete mode 100644 identify/identify_test.go
 delete mode 100644 identify/message.proto
 create mode 100644 importer/rabin.go
 delete mode 100644 importer/split_test.go
 rename {swarm => net/conn}/conn.go (66%)
 rename {swarm => net/conn}/conn_test.go (86%)
 create mode 100644 net/interface.go
 create mode 100644 net/message/message.go
 create mode 100644 net/mux/Makefile
 create mode 100644 net/mux/mux.go
 create mode 100644 net/mux/mux.pb.go
 create mode 100644 net/mux/mux.proto
 create mode 100644 net/mux/mux_test.go
 create mode 100644 net/net.go
 create mode 100644 net/net_test.go
 create mode 100644 net/service/Makefile
 create mode 100644 net/service/request.go
 create mode 100644 net/service/request.pb.go
 create mode 100644 net/service/request.proto
 create mode 100644 net/service/request_test.go
 create mode 100644 net/service/service.go
 create mode 100644 net/service/service_test.go
 create mode 100644 net/swarm/conn.go
 create mode 100644 net/swarm/swarm.go
 rename {swarm => net/swarm}/swarm_test.go (56%)
 create mode 100644 peer/peerstore.go
 create mode 100644 peer/peerstore_test.go
 create mode 100644 peer/queue/distance.go
 create mode 100644 peer/queue/interface.go
 create mode 100644 peer/queue/queue_test.go
 create mode 100644 peer/queue/sync.go
 create mode 100644 routing/dht/Makefile
 create mode 100644 routing/dht/handlers.go
 create mode 100644 routing/dht/query.go
 create mode 100644 routing/keyspace/keyspace.go
 create mode 100644 routing/keyspace/xor.go
 create mode 100644 routing/keyspace/xor_test.go
 delete mode 100644 swarm/interface.go
 delete mode 100644 swarm/mes_listener.go
 delete mode 100644 swarm/mes_listener_test.go
 delete mode 100644 swarm/mes_wrapper.pb.go
 delete mode 100644 swarm/mes_wrapper.proto
 delete mode 100644 swarm/swarm.go
 delete mode 100644 swarm/wrapper.go
 create mode 100644 util/key_set.go
 create mode 100644 util/testutil/blocks.go
 create mode 100644 util/todocounter/counter.go
First, rewinding head to replay your work on top of it...
Fast-forwarded master to 676d9c5c968c8ab839f8f3f0f41fc047eab86bc6.

@whyrusleeping
Copy link
Member

@jbenet o.o that changelist....

@btc btc deleted the net branch September 23, 2014 21:09
@aschmahmann aschmahmann mentioned this pull request Sep 22, 2020
72 tasks
ariescodescream pushed a commit to ariescodescream/go-ipfs that referenced this pull request Oct 23, 2021
ariescodescream pushed a commit to ariescodescream/go-ipfs that referenced this pull request Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants