Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Permutations #790

Merged

Conversation

Philippe-Cholet
Copy link
Member

Fixes #747

This is quite a rewrite, very little is untouched.
count and size_hint are really simplified with the new PermutationState::size_hint_for.

cargo bench permutation
// Before this PR
permutations iter       [62.130 µs 62.355 µs 62.620 µs]
permutations range      [61.637 µs 61.850 µs 62.104 µs]
permutations slice      [64.177 µs 64.314 µs 64.459 µs]
// Now
permutations iter       [57.280 µs 57.478 µs 57.702 µs]    -8.4%
permutations range      [59.714 µs 59.914 µs 60.138 µs]    -4.7%
permutations slice      [59.584 µs 59.720 µs 59.868 µs]    -7.1%

I did not expect any performance improvement, but nice to have a little one.

Copy link
Member

@phimuemue phimuemue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Philippe-Cholet, looks good to me. The flattened structure seems much nicer.

This took me a while to review, and I could only do it by "replaying" what you possibly did yourself (see d188e8e...phimuemue:rust-itertools:permutations-states-2, the end result being exactly what you proposed). Unfortunately, I am not aware of a process that would allow quicker reviews apart from more fine-grained commits.

Maybe it's worth preserving the history of fine-grained transformations. Please use your judgment, and either merge this or take my commits and merge them.

src/permutations.rs Outdated Show resolved Hide resolved
}
let Self { vals, state } = self;
let n = vals.count();
state.size_hint_for(n).1.unwrap()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, we go via SizeHint (i.e. (usize, Option<usize>)) to compute count. Fine given the simplification we gain by this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, size_hint_for mostly ends with (x.unwrap_or(usize::MAX), x) on which we either do .0 or .1.
At worst, it needlessly unwrapped an option one time.

fn size_hint_for(&self, n: usize) -> SizeHint {
// At the beginning, there are `n!/(n-k)!` items to come.
let at_start = |n, k| {
debug_assert!(n >= k);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you shortly explain why this debug_assert holds?

Copy link
Member Author

@Philippe-Cholet Philippe-Cholet Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote that in my previous branch some weeks ago so I was not much familiar with it, and take a glance at it is definitely not enough as it requires a bit of thinking.

This debug_assert! only occurs with Start and Buffered variants.
At definition, we prefill the lazy buffer with k values. It has enough values (or we would have the End variant) so vals.len() >= k (vals.len()==k at definition, more later).
size_hint_for is then called with:

  • in the case of count: n = vals.count() >= vals.len() (see lazy buffer for >=) ;
  • in the case of size_hint: n = vals.size_hint().0 >= vals.len() (see lazy buffer).
    Similar for n = vals.size_hint().1.

So in each case: n >= vals.len() >= k. Basically, it holds because we prefilled with k values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I'm considering to soon work on making all our iterators lazy (such as #602) and I'll surely turn that assertion into if n < k { return (0, Some(0)); } (and move the "prefill the lazy buffer" part).

Comment on lines +145 to +146
low = self.state.size_hint_for(low).0;
upp = upp.and_then(|n| self.state.size_hint_for(n).1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern seems familiar to me... Do you know if it occurs somewhere else? If so, should we introduce size_hint::map? (We can do this separately, it just occured to me.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern occurs most of the time I think. I thought of a similar method in a messy PR but I had headaches about the conditions on f for the resulting size hint to be correct, so as I wrote several size hints I went with applying the pattern manually.

@Philippe-Cholet
Copy link
Member Author

Philippe-Cholet commented Nov 1, 2023

@phimuemue
I'm sorry you had some hard time reviewing it. I definitely tried and struggled to make more fine-grained commits here. But I should have thought of your very helpful intermediary Loaded(CompleteState) -> LoadedStart, LoadedOngoing.
I'm a work in progress (especially git), and to improve myself I splitted it again myself, trying to leave the last commit intact (FusedIterator, but I git-failed so I had to write it again).

You can check that there is no difference between our branches:
https://github.com/Philippe-Cholet/itertools/compare/permutations-states-2..phimuemue:rust-itertools:permutations-states-2 (I've just learnt that .. and ... have different meaning in this url).

@phimuemue
Copy link
Member

@phimuemue I'm sorry you had some hard time reviewing it. I definitely tried and struggled to make more fine-grained commits here. But I should have thought of your very helpful intermediary Loaded(CompleteState) -> LoadedStart, LoadedOngoing. I'm a work in progress (especially git), and to improve myself I splitted it again myself, trying to leave the last commit intact (FusedIterator, but I git-failed so I had to write it again).

You can check that there is no difference between our branches: https://github.com/Philippe-Cholet/itertools/compare/permutations-states-2..phimuemue:rust-itertools:permutations-states-2 (I've just learnt that .. and ... have different meaning in this url).

No worries. I hope it didn't come across an offense. Thanks again for taking the time to simplify this.

@phimuemue phimuemue added this pull request to the merge queue Nov 1, 2023
Merged via the queue into rust-itertools:master with commit 14d2fa2 Nov 1, 2023
8 checks passed
@Philippe-Cholet Philippe-Cholet deleted the permutations-states-2 branch November 1, 2023 14:32
@Philippe-Cholet
Copy link
Member Author

@phimuemue I did not see any offense, only the opportunity to improve the way I write commits.

@jswrenn jswrenn mentioned this pull request Nov 14, 2023
@jswrenn jswrenn added this to the next milestone Nov 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Permutations unnecessarily complex?
3 participants