Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent sending of ACK-only packets #1130

Merged
merged 1 commit into from
Jun 22, 2021
Merged

Conversation

Matthias247
Copy link
Contributor

The priority-based send state implementation had an issue where if returned
self.can_send() == true even if no data was available for sending.
This condition lead the poll_transmit method to initiate a transmission which
passed the ACK-only check since it assumed there was stream data to send.
However when trying to write stream frames no actual stream data could be
written.

The reason for this was that the can_send() condition only checked for the
length of the binary heap. However the length of this one was only reduced
in write_stream_frames() when the level was inspected the next time, and
not after data of one level was actually written.

This change fixes that, and drops the unused levels immediately after data
was written. There is on exception however: If only one level is left, it is
kept around and reused for future transmits to avoid deallocating and
reallocating the transmit queue continuously. However the can_send check
was updated to account for the empty level.

I also added some additional debug asserts which cross-checks
if an operation which announced to write more than just ACKs
still ended up only writing ACKs. That should make it easier to
determine similar issues in the future.

Ack only transmissions in benchmark before change: 1462

Ack only transmissions in benchmark after change: 45

quinn-proto/src/connection/mod.rs Outdated Show resolved Hide resolved
quinn-proto/src/connection/streams/state.rs Outdated Show resolved Hide resolved
@Matthias247
Copy link
Contributor Author

Lints fail due to rustc 1.53 update. Should be fixed with #1147

Copy link
Collaborator

@Ralith Ralith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM aside from a readability issue and the lints.

quinn-proto/src/connection/streams/mod.rs Outdated Show resolved Hide resolved
quinn-proto/src/connection/mod.rs Outdated Show resolved Hide resolved
@Matthias247 Matthias247 force-pushed the pending_fix branch 3 times, most recently from 0d0956d to a49564a Compare June 22, 2021 02:52
Ralith
Ralith previously approved these changes Jun 22, 2021
Copy link
Member

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Some nits:

quinn-proto/src/connection/mod.rs Outdated Show resolved Hide resolved
quinn-proto/src/connection/mod.rs Outdated Show resolved Hide resolved
Comment on lines 316 to 317
if pending.len() == 1 {
let mut first = pending.peek_mut().unwrap();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we could do if let Some(first) = pending.peek_mut() or similar here, avoiding the unwrap?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -295,7 +295,10 @@ impl StreamsState {
}

pub fn can_send(&self) -> bool {
!self.pending.is_empty()
match self.pending.peek() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe unwrap_or() here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced it with

self.pending
            .peek()
            .map(|head| !head.queue.borrow().is_empty())
            .unwrap_or(false)

but I actually don't find it better, and I feel I gotten the advice to move to a match in the past

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was actually thinking of map_or. Does that help?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated with that. Saves at least I line, but I overall still say those are all the same to me 🤷🏻‍♂️

Comment on lines +489 to +492
// Enqueue for a different level. If the current level is empty, drop it
if level.queue.borrow().is_empty() && num_levels != 1 {
// We keep the last level around even in empty form so that
// the next insert doesn't have to reallocate the queue
PeekMut::pop(level);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we really should find a way to avoid the three duplicated code sections.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the priorities again or don't care about performance :-)

While I think there might be ways to make this prettier, I'm not super inclined to investigate further in this. This is a rather performance-impacting bug that is open for 30 days, and I would rather like to see it fixed ASAP and spend energy into more impactful things than moving code around.

It's also not exactly duplicated.

The priority-based send state implementation had an issue where if returned
`self.can_send() == true` even if no data was available for sending.
This condition lead the `poll_transmit` method to initiate a transmission which
passed the ACK-only check since it assumed there was stream data to send.
However when trying to write stream frames no actual stream data could be
written.

The reason for this was that the `can_send()` condition only checked for the
length of the binary heap. However the length of this one was only reduced
in `write_stream_frames()` when the level was inspected the next time, and
not after data of one level was actually written.

This change fixes that, and drops the unused levels immediately after data
was written. There is on exception however: If only one level is left, it is
kept around and reused for future transmits to avoid deallocating and
reallocating the transmit queue continuously. However the `can_send` check
was updated to account for the empty level.

I also added some additional debug asserts which cross-checks
if an operation which announced to write more than just ACKs
still ended up only writing ACKs. That should make it easier to
determine similar issues in the future.

**Ack only transmissions in benchmark before change:** 1462

**Ack only transmissions in benchmark after change:** 45
@djc djc merged commit 3a30aa1 into quinn-rs:main Jun 22, 2021
@Matthias247 Matthias247 deleted the pending_fix branch September 15, 2022 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants