Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data splitting and merging #260

Open
89trillion-wuyanqing opened this issue Mar 29, 2024 · 7 comments
Open

Data splitting and merging #260

89trillion-wuyanqing opened this issue Mar 29, 2024 · 7 comments

Comments

@89trillion-wuyanqing
Copy link

89trillion-wuyanqing commented Mar 29, 2024

I have implemented the server and client using the kcp-go framework. The server sent a data packet larger than 1500 bytes. I expected the server to split this packet, and then the client would merge it upon return. However, the result was that the server split the packet into multiple packets, and the client ended up receiving the fragmented packets, rendering them unusable.

In your WriteBuffers method, you first split the sent data according to the maximum transmission unit parameter (mss) and pass the segmented byte data to the Send method. Then, in the Send method, the logic of splitting the data into fragments is applied. As a result, the data packet will never exceed the maximum transmission unit, and therefore, the logic of splitting and merging will never be triggered.

I believe there needs to be a modification in the framework. At the user level, after sending a data packet, the receiving end should receive a complete packet instead of it being split into multiple fragments.

Therefore, I suggest that the WriteBuffers method does not need to split the sent data according to the maximum transmission unit parameter (mss). Instead, the original data should be passed directly to the Send method.

I'm not a native English speaker, so my expression may not be very accurate. Please understand.

Snipaste_2024-03-29_23-27-53
@xtaci
Copy link
Owner

xtaci commented Apr 2, 2024

You can encode like this: |size|payload| , try to add a size prefix before the payload.

@89trillion-wuyanqing
Copy link
Author

I don't quite understand what you mean, can you tell in which method this parameter should be set.

@xtaci
Copy link
Owner

xtaci commented May 20, 2024

why not just add a prefix before the data, the prefix suggests how many bytes are there in the packet.
|size|payload|size|payload|.... |size|payload|

@89trillion-wuyanqing
Copy link
Author

Do you mean that before calling the Write method, I split the binary data myself, and each split data packet needs to add a 2-byte prefix to indicate the size of the data packet?
Snipaste_2024-05-20_22-01-23

In fact, my core question is whether the work of splitting data packets is implemented in the third-party package or by the user himself?

@xtaci
Copy link
Owner

xtaci commented May 20, 2024

users should be in charge of managing the border of packets. In general, kcp-go provides a realiable stream only.

@89trillion-wuyanqing
Copy link
Author

When I looked at the code, I found that you would split the large data packets into segments, so I thought that the third-party package had already encapsulated the data splitting work.
Snipaste_2024-05-21_21-56-49

Because after I modified this code, when sending a large data packet, the simulated client can normally receive a complete assembled data packet, so I think that this data splitting work needs to be encapsulated, and the user is not responsible for managing the boundaries of the data packet.
Snipaste_2024-03-29_23-27-53

So if I make the above changes, will there be any serious consequences?

@xfnyl
Copy link

xfnyl commented Jul 25, 2024

I encountered the same problem as you, and my solution is the same as yours, which is to modify the WriteBuffers() method so that it does not split packets before calling kcp.Send().
I think if streaming data is sent, the client will automatically combine the packets, and it will not matter whether it is changed or not. But if it is a single large packet that exceeds the MSS, the client will not automatically combine the packets, so this processing method is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants