Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

extend simple peer (not peerjs, oops) to handle buffered/packet transmission; add raw dependency w/MIT license #25

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

disarticulate
Copy link
Contributor

Closes #20

  • Extends the simple peer class to override the send and _onChannelMessage functions to handle chunked data packets using custom defined packets encodePacket and decodePacket
  • Adds bundled & minified (maybe you want to include it somewhere else) dependency to use integers for header details (8 bytes) as discussed in the issue, numbering packets.

The custom packet setup could probably be simplfied as we're counting each full data (txOrd), each packet sent (index), how many packets (length), the size of the packet's data (chunkSize) and the total size of the data (totalSize)

Ran the tests and get some typescript errors that come from the SimplePeerExtended extends Peer where we're using something from the parent class that isnt available. Also, the minified dependency would either need to be brought in officially, or ignored.

Copy link
Member

@dmonad dmonad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @disarticulate, thanks so much for this!

I like the approach that you have taken. But I'm a bit hesitant to release a new major release of y-webrtc with these breaking changes. However, I think it should be possible for us to make this change non-breaking by making it optional whether we adapt your polyfill (or another one which might fix other issues).


encodePacket ({ chunk, txOrd, index, length, totalSize, chunkSize }) {
const encoded = concatenate(Uint8Array, [
new Uint8Array(new Int64BE(txOrd).toArrayBuffer()), // 8 bytes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the reason why you don't use BigUint64Array is that it is not yet supported in Safari.

I developed an efficient encoder exactly for this problem: https://github.com/dmonad/lib0/blob/main/encoding.js

import * as encoding from 'lib0/encoding'
import * as decoding from 'lib0/decoding'

// example of encoding unsigned integers and strings efficiently

const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256) 
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)

const decoder = new decoding.createDecoder(buf) 
decoding.readVarUint(decoder) // => 256 
decoding.readVarString(decoder) // => 'Hello world!' 
decoding.hasContent(decoder) // => false - all data is read

The documentation for other encoding techniques is here: https://github.com/dmonad/lib0

But I realize that this is already working in the current state. But maybe we can avoid pulling in more dependencies that are superflous.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a header, we need to pad the values. Note the // 8 bytes comments I've mocked up a replacement class like:

class Int64 {
  constuctor (int64) {
    this.encoder = new encoding.createEncoder()
    encoding.writeVarUint(encoder, int64) 
  }
  toArrayBuffer () {
    return encoding.toUint8Array(this.encoder)
  }
}

However, when I test the recommended dependency, we get:

...
encoding.writeVarUint(encoder, 1)
encoding.toUint8Array(encoder)
> Uint8Array(1) [ 1 ]

its not obvious to me how to do pad/unpad safely, so the whole decode/encode would need a rewrite

import Peer from 'simple-peer/simplepeer.min.js'

// import Peer from 'simple-peer/simplepeer.min.js'
import Peer from './SimplePeerExtended'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting. So you implemented a layer around simple-peer that handles this issue.

Would it be possible that you publish a separate package that we can include as a polyfill for the default implementation? This is already done in y-websocket where we define the WebSocket as an argument.

new WebsocketProvider(URL, room, { WebSocket: MyCustomWebsocketPolyfill })

We could do something similar to y-webrtc without breaking the existing API.

new WebrtcProvider(room, yjs, { SimplePeer: SimplePeerExtended })

I'd prefer that approach because it allows us to test out your implementation before breaking everyone else's existing deployments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i can publish a module and put something up on github, but it seems like you'd need move SimplePeer to a peerDependency to avoid duplicating code, otherwise your library is pulling in Peer while your user is also pulling in a modified peer.

Also, it looks like WebrtcConn is what's using the Peer and that's being called from SignalingConn...etc...

I can't figure out how you'd thread that option into the proper slot because it call comes from a global import Peer from 'simple-peer/simplepeer.min.js'

@datakurre
Copy link

@disarticulate Is this patch the latest version you've been applying? I see errors when clients disconnect. Those seem to go away, if I include super.destroy() from you main branch.

Beyond that, a supported way to apply customized SimplePeer would be great 🙏.

@disarticulate
Copy link
Contributor Author

@disarticulate Is this patch the latest version you've been applying? I see errors when clients disconnect. Those seem to go away, if I include super.destroy() from you main branch.

Beyond that, a supported way to apply customized SimplePeer would be great 🙏.

Yes. I haven't been developing with it recently, and haven't followed any yjs updates since this was proposed.

check https://github.com/disarticulate/y-webrtc for when it was developed. it looks like simple-peer would need be moved to a peer dependency. I think the behavior is stable, but how it should be integrated takes some considerations that I haven't returned to.

@disarticulate
Copy link
Contributor Author

disarticulate commented Jul 30, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implment Packet buffer for sending
3 participants