Basic spec compliance needed to get a node-ipfs running #13
Comments
|
@jbenet "candidate attack plan" from the meeting notes is ready :) |
|
@diasdavid candidate attack plan looks great to me!
|
|
From @jbenet on IRC
Agree, I've put myself doing "Get go-ipfs and node-ipfs talking", it will help pave the path for the remaining tasks and how to partition them. |
|
Great! 👍 |
|
we can probably assign the rest once we have a clearer scope. also, i think @krl began working on the node merkledag. |
|
For basic wire protocol interop, all we need is (lots of steps but each one is relatively small):
(could actually talk without spdy first) |
|
@whyrusleeping have time this week to take on any of the go things above? (part the spdy part?) |
|
@diasdavid @mappum @krl @travisperson -- what things from the wire interop list might you be able to take this week? if we had a thing by the end of the week that spoke the wire protocol, it would be a great start. |
|
(and anyone else watching, feel free to help! \o/) |
|
After thesis delivery on Friday morning, my intent is to to focus entirely on:
I'll be online, so anyone that wants to work at the same time in this stuff, just let me know |
|
@jbenet @diasdavid in multistream, is the newline included in the length? (if so i more or less already have that module done) |
|
@jbenet i was going to work on the records ipns stuff, but i can do this instead. |
|
@mafintosh yes, the newline is included in the length. what's your module? |
|
ill take on "write go-multistream and go-multistream-select (may be just one pkg)" |
|
@whyrusleeping edit my message and sign yourself up? + link to relevant issues there as they're made? |
|
@jbenet do we want multistream to be aware if the stream is binary/plain text? If we have, is the binary version already have a defined expected length? (To know how much bytes should be reserved for the length in the beginning) |
|
Multistream itself shouldn't care-- it just defines the header. As for the expected length, it's a varint, but for now the go impl uses 'encoding/binary', which bounds it to a uint64 maximum (larger varints are treated by the implementation as errors, so the functionality is there to grow, but we can do the easy safe thing for now). Also I need to adjust the docs for multistream-select -- @whyrusleeping and I adjusted some things. (Made "ls\n" and "na\n" and ls output messages varint prefixed too) @whyrusleeping imemented: - https://github.com/whyrusleeping/go-multistream/ - https://github.com/whyrusleeping/mss-nc So take a look at those meanwhile? — On Fri, Jun 5, 2015 at 10:38 AM, David Dias notifications@github.com
|
|
(Also we should make sure whether the encoding/binary varint is what we want. I'd like a flavor of varint that means "<127" is the same byte value (I.e. Highest order bit is continuation), so impls can check easily) — On Fri, Jun 5, 2015 at 10:38 AM, David Dias notifications@github.com
|
|
update:
I believe that in parallel to figuring out what is missing to full interop, I can start looking on having spdy on top of this, so then it is all of the goodness from routing layer and above :) |
great! 👍
So this is for There's already an ack, but the ack "confirms the protocol name", and is also the same thing as starting a conversation, too: If we added an Without this extra ACK, you could have a totally one-sided conversation -- meaning a push-stream that is not duplex, without needing to send anything back: # example of me sending stuff to a silent listener
nc localhost 9090
> /ipfs/QmdRK.../multistream-select/0.3.0 # let's speak multistream-select/0.3.0
> /ipfs/QmVXZ.../ipfs-dht/0.2.3 # i'm going to speak ipfs-dht/0.2.3
> <dht-message>
> <dht-message>
> <dht-message>
# example of me dialing an endpoint that just outputs a message and exits
nc localhost 9090
< /ipfs/QmdRK.../multistream-select/0.3.0 # let's speak multistream-select/0.3.0
< /ipfs/QmBBQ.../time/0.1.2. # i'm going to speak time/0.1.2
< 2015-06-07 14:32:11Not sure if this makes it clear, hard to explain. Anyway, this makes it useful for protocols that are simple information sources/sinks, or aren't "online" (e.g. "offline protocols" are those where you dont need to maintain a synchronous connection). So a "consumer" of the protocol can just encounter a partial stream like: And the "consumer" can consume this without needing to talk synchronously with whoever the "producer" was.
oh dang, name clash. d-d-d-double name clash (https://www.npmjs.org/package/multistream). We can also call this pkg https://www.npmjs.com/package/multistream-select or some variation on
yay! \o/
Hmm, interesting.
Sorry, my mistake for not making this clearer. I didn't mean to imply a requirement for an ordering of "who states the protocol first". I think that in order to support "one-sided conversations" it's up to both to find agreement. In some situations, you want the dialer to poke the server by opening a connection, and the server says "hey, I speak multistream-select/0.1.2, buddy", and have the client (who's dialing anyway) adjust to the server's demands. In other cases, the clients could be very dumb + constrained, so they'll just speak out one protocol, and the server can be lenient. We observe cases like this all over, so I sort of see this as a "semi-synchronous" protocol, where the
Awesome! \o/ |
|
Thank you @jbenet, that was indeed a good explanation! I've updated the module (and published it to make sure we don't loose that name), categorising the two modes of working as |
|
ok, got what was the problem with the interop problems I was having between go and node multistream impls. It seems that sorribas impl of length-prefixed-message doesn't include a '\n', and since we changed the protocol to include it, well, it barfed :). I'll just add the '\n' myself to the buffer that is passed onto lpm when a write occurs and trim it for the reads :). On it :) |
|
TODO:
|
|
Oh, how time flies :) June 1, feels just like yesterday :) |
minimum spec
validate the usefulness of code already created (deprecate/rebuild/good)
The go is to go through these modules, understand what stays, what leaves and what is missing compared to the go-ipfs. (probably will draw some ASCII art after I get this into my head, sometimes helps to think and communicate better)
candidate attack plan:
The text was updated successfully, but these errors were encountered: