remove limit to outgoing msgs

[jbenet]

currently, if a message is greater than net/conn.MaxMessageSize, it is simply dropped. the network layer should be able to handle chunking (this may just work out of the box, given we're only using stream connections atm.

[whyrusleeping]

The receiving side needs to know how many bytes to read

[whyrusleeping]

Actually, now thati think about it, we send the length. One thing we would have to though, is make sure we know exactly where buffers stop being used. Otherwise we have a memory allocation nightmare.

[jbenet]

We can still use the memory pool, we can just allocate larger buffers when needed. Not sure if the pool does this but we could also free buffers if there is a high water mark

—
Sent from Mailbox

On Mon, Nov 24, 2014 at 11:10 PM, Jeromy Johnson notifications@github.com
wrote:

Actually, now thati think about it, we send the length. One thing we would have to though, is make sure we know exactly where buffers stop being used. Otherwise we have a memory allocation nightmare.

Reply to this email directly or view it on GitHub:
#384 (comment)

[whyrusleeping]

The sync.Pool is garbage collector aware, if it contains too many extra buffers, they will be freed during the next GC

[jbenet]

nice.

[whyrusleeping]

I vote on bumping up the priority of a msgio refactor.

[jbenet]

Closed in #406

[btc]

There's still a limit on message size (albeit a large one), yeah?

[jbenet]

@maybebtc yes, but it's desired. we don't want to be sending massive messages. It signals massive blocks, which we don't want.

that reminds me, @whyrusleeping unixfs needs indirect blocks of arbitrary depth. right now, on massive files, the indirect block becomes huge. (#420)

[btc]

@maybebtc yes, but it's desired. we don't want to be sending massive messages. It signals massive blocks, which we don't want.

Are clients made aware of the limit?

[jbenet]

Are clients made aware of the limit?

nope :/ netchan sounds good at first till you actually build it.