-
-
Notifications
You must be signed in to change notification settings - Fork 116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
extend simple peer (not peerjs, oops) to handle buffered/packet transmission; add raw dependency w/MIT license #25
base: master
Are you sure you want to change the base?
Conversation
…ency with license
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @disarticulate, thanks so much for this!
I like the approach that you have taken. But I'm a bit hesitant to release a new major release of y-webrtc with these breaking changes. However, I think it should be possible for us to make this change non-breaking by making it optional whether we adapt your polyfill (or another one which might fix other issues).
|
||
encodePacket ({ chunk, txOrd, index, length, totalSize, chunkSize }) { | ||
const encoded = concatenate(Uint8Array, [ | ||
new Uint8Array(new Int64BE(txOrd).toArrayBuffer()), // 8 bytes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess the reason why you don't use BigUint64Array is that it is not yet supported in Safari.
I developed an efficient encoder exactly for this problem: https://github.com/dmonad/lib0/blob/main/encoding.js
import * as encoding from 'lib0/encoding'
import * as decoding from 'lib0/decoding'
// example of encoding unsigned integers and strings efficiently
const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256)
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)
const decoder = new decoding.createDecoder(buf)
decoding.readVarUint(decoder) // => 256
decoding.readVarString(decoder) // => 'Hello world!'
decoding.hasContent(decoder) // => false - all data is read
The documentation for other encoding techniques is here: https://github.com/dmonad/lib0
But I realize that this is already working in the current state. But maybe we can avoid pulling in more dependencies that are superflous.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For a header, we need to pad the values. Note the // 8 bytes comments
I've mocked up a replacement class like:
class Int64 {
constuctor (int64) {
this.encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, int64)
}
toArrayBuffer () {
return encoding.toUint8Array(this.encoder)
}
}
However, when I test the recommended dependency, we get:
...
encoding.writeVarUint(encoder, 1)
encoding.toUint8Array(encoder)
> Uint8Array(1) [ 1 ]
its not obvious to me how to do pad/unpad safely, so the whole decode/encode would need a rewrite
import Peer from 'simple-peer/simplepeer.min.js' | ||
|
||
// import Peer from 'simple-peer/simplepeer.min.js' | ||
import Peer from './SimplePeerExtended' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is interesting. So you implemented a layer around simple-peer that handles this issue.
Would it be possible that you publish a separate package that we can include as a polyfill for the default implementation? This is already done in y-websocket where we define the WebSocket
as an argument.
new WebsocketProvider(URL, room, { WebSocket: MyCustomWebsocketPolyfill })
We could do something similar to y-webrtc without breaking the existing API.
new WebrtcProvider(room, yjs, { SimplePeer: SimplePeerExtended })
I'd prefer that approach because it allows us to test out your implementation before breaking everyone else's existing deployments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i can publish a module and put something up on github, but it seems like you'd need move SimplePeer to a peerDependency to avoid duplicating code, otherwise your library is pulling in Peer while your user is also pulling in a modified peer.
Also, it looks like WebrtcConn
is what's using the Peer
and that's being called from SignalingConn
...etc...
I can't figure out how you'd thread that option into the proper slot because it call comes from a global import Peer from 'simple-peer/simplepeer.min.js'
@disarticulate Is this patch the latest version you've been applying? I see errors when clients disconnect. Those seem to go away, if I include Beyond that, a supported way to apply customized SimplePeer would be great 🙏. |
Yes. I haven't been developing with it recently, and haven't followed any yjs updates since this was proposed. check https://github.com/disarticulate/y-webrtc for when it was developed. it looks like simple-peer would need be moved to a peer dependency. I think the behavior is stable, but how it should be integrated takes some considerations that I haven't returned to. |
The boring answer is the encoder selected is encoding 64 bit integers.
The arbitrary answer is it made the header simple to decode and encode.
Someone could rightsize it all, some of those values are purely redundant.
I considered a system that did packet routing so wanted space.
…On Thu, Jul 28, 2022, 17:14 Vitor de Mello Freitas ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In src/SimplePeerExtended.js
<#25 (comment)>:
> +}
+
+class SimplePeerExtended extends Peer {
+ constructor (opts) {
+ super(opts)
+ this._opts = opts
+ this._txOrdinal = 0
+ this._rxPackets = []
+ this._txPause = false
+ this.webRTCMessageQueue = []
+ this.webRTCPaused = false
+ }
+
+ encodePacket ({ chunk, txOrd, index, length, totalSize, chunkSize }) {
+ const encoded = concatenate(Uint8Array, [
+ new Uint8Array(new Int64BE(txOrd).toArrayBuffer()), // 8 bytes
@disarticulate <https://github.com/disarticulate> Why do the values need
to be padded to 8 bytes in the header?
—
Reply to this email directly, view it on GitHub
<#25 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEFHWPLDSXMKA2FL3GAULI3VWMA2TANCNFSM45JCHYTA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Closes #20
send
and_onChannelMessage
functions to handle chunked data packets using custom defined packetsencodePacket
anddecodePacket
The custom packet setup could probably be simplfied as we're counting each full data (
txOrd
), each packet sent (index
), how many packets (length
), the size of the packet's data (chunkSize
) and the total size of the data (totalSize
)Ran the tests and get some typescript errors that come from the
SimplePeerExtended extends Peer
where we're using something from the parent class that isnt available. Also, the minified dependency would either need to be brought in officially, or ignored.