Implement nth_back for slice::{Iter, IterMut} #60772

timvermeulen · 2019-05-12T22:39:16Z

Part of #54054.

I implemented nth_back as straightforwardly as I could, and then slightly changed nth to match nth_back. I believe I did so correctly, but please double-check 🙂

I also added the helper methods zst_shrink, next_unchecked, and next_back_unchecked to get rid of some duplicated code. These changes hopefully make this code easier to understand for new contributors like me.

I noticed the is_empty! and len! macros which sole purpose seems to be inlining, according to the comment right above them, but the is_empty and len methods are already marked with #[inline(always)]. Does that mean we could replace these macros with method calls, without affecting anything? I'd love to get rid of them.

rust-highfive · 2019-05-12T22:39:27Z

r? @TimNN

(rust_highfive has picked a reviewer for you, use r? to override)

RalfJung · 2019-05-13T11:18:56Z

src/libcore/slice/mod.rs

+            // end of the iterator backwards by `n`.
+            #[inline(always)]
+            fn zst_shrink(&mut self, n: isize) {
+                self.end = (self.end as * $raw_mut u8).wrapping_offset(-n) as * $raw_mut T;


Maybe add a debug_assert making sure mem::size_of::<T>() == 0?

I didn't do this as not to affect the performance of debug builds, although I'm not opposed to adding it. Either way, this function should probably be unsafe?

RalfJung · 2019-05-13T11:20:44Z

src/libcore/slice/mod.rs

+                        accum = f(accum, self.next_unchecked())?;
+                        accum = f(accum, self.next_unchecked())?;
+                        accum = f(accum, self.next_unchecked())?;
+                        accum = f(accum, self.next_unchecked())?;


Last time I touched these, even introducing inline(always) functions in some of these places had significant impact on performance. This should get a perf run, and ideally also some manual benchmarking.

Yikes, yep, benchmarks I just ran confirm this. So are they not actually being inlined, or what's going on?

So we don't need to bother perfbot then?^^

So are they not actually being inlined, or what's going on?

Your guess is as good as mine. One hypothesis we had last time was that inlining happens after some other passes -- so manually inlined code basically is already present earlier in the pipeline and gets simplified better.

You could try to compile your benchmark to LLVM IR with the old and new version of libstd and compare. You'll see that the function did get inlined, but the IR is still vastly different.

It's unfortunate that functions apparently aren't zero-cost here, but I guess it is what it is.

I turned next_unchecked and next_back_unchecked into macros and that seems to have fixed all the regressions. 🙂 It's hard to say for sure though, every time I run the benchmarks there are a few that change by up to 5% or so in either direction, although it's not always the same ones. It seems like they're just not very stable.

RalfJung · 2019-05-13T11:26:14Z

For perf:
@bors try

I noticed the is_empty! and len! macros which sole purpose seems to be inlining, according to the comment right above them, but the is_empty and len methods are already marked with #[inline(always)]. Does that mean we could replace these macros with method calls, without affecting anything? I'd love to get rid of them.

When I tried, this made a difference. See in particular the discussion starting here.

bors · 2019-05-13T11:26:27Z

⌛ Trying commit dde6587388afb05ddf716c97aaa6d8cdfecfae97 with merge a7f658d531f58408d872991a0aad92876b86712c...

bors · 2019-05-13T13:47:46Z

☀️ Try build successful - checks-travis
Build commit: a7f658d531f58408d872991a0aad92876b86712c

rust-highfive · 2019-05-13T16:30:40Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:0545aa4a:start=1557764063421599771,finish=1557764259021378718,duration=195599778947
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[00:04:21] tidy error: /checkout/src/libcore/slice/mod.rs:3030: trailing whitespace
[00:04:26] some tidy checks failed
[00:04:26] 
[00:04:26] 
[00:04:26] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:26] 
[00:04:26] 
[00:04:26] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:26] Build completed unsuccessfully in 0:01:12
[00:04:26] Build completed unsuccessfully in 0:01:12
[00:04:26] make: *** [tidy] Error 1
[00:04:26] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:268c05cc
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Mon May 13 16:22:14 UTC 2019
---
travis_time:end:12cb7b00:start=1557764535551006343,finish=1557764535556284451,duration=5278108
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:014669a0
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:10974a64
travis_time:start:10974a64
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:13c8870b
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

timvermeulen · 2019-05-13T17:15:51Z

@RalfJung Thanks! That's very helpful. Looks like we can't get rid of the macros yet, then.

RalfJung · 2019-05-13T19:03:57Z

Let's see what perf has to say, then.

@bors try

bors · 2019-05-13T19:04:09Z

⌛ Trying commit 3834d0e with merge e89b9fd...

Implement nth_back for slice::{Iter, IterMut} Part of #54054. I implemented `nth_back` as straightforwardly as I could, and then slightly changed `nth` to match `nth_back`. I believe I did so correctly, but please double-check 🙂 I also added the helper methods `zst_shrink`, `next_unchecked`, and `next_back_unchecked` to get rid of some duplicated code. These changes hopefully make this code easier to understand for new contributors like me. I noticed the `is_empty!` and `len!` macros which sole purpose seems to be inlining, according to the comment right above them, but the `is_empty` and `len` methods are already marked with `#[inline(always)]`. Does that mean we could replace these macros with method calls, without affecting anything? I'd love to get rid of them.

bors · 2019-05-13T21:23:04Z

☀️ Try build successful - checks-travis
Build commit: e89b9fd

RalfJung · 2019-05-14T07:41:31Z

@rust-timer build e89b9fd

rust-timer · 2019-05-14T07:41:32Z

Success: Queued e89b9fd with parent a9ec99f, comparison URL.

timvermeulen · 2019-05-14T20:04:29Z

It's not clear to me whether the 0.8% change to clap-rs-debug is an actual regression, or just noise... It seems like a slight regression overall if I naively add up all the percentages. Maybe I should turn zst_shrink into a macro, too? That's the only possible reason for regressions that I can see here.

timvermeulen · 2019-05-24T23:53:29Z

I also made zst_shrink into a macro, in order to make all code equivalent to the code on master after all macros are expanded.

@bors try

bors · 2019-05-24T23:53:30Z

@timvermeulen: 🔑 Insufficient privileges: not in try users

RalfJung · 2019-05-25T08:05:23Z

@bors try

bors · 2019-05-25T08:05:36Z

⌛ Trying commit 97a6c93 with merge 01fe51e...

Implement nth_back for slice::{Iter, IterMut} Part of #54054. I implemented `nth_back` as straightforwardly as I could, and then slightly changed `nth` to match `nth_back`. I believe I did so correctly, but please double-check 🙂 I also added the helper methods `zst_shrink`, `next_unchecked`, and `next_back_unchecked` to get rid of some duplicated code. These changes hopefully make this code easier to understand for new contributors like me. I noticed the `is_empty!` and `len!` macros which sole purpose seems to be inlining, according to the comment right above them, but the `is_empty` and `len` methods are already marked with `#[inline(always)]`. Does that mean we could replace these macros with method calls, without affecting anything? I'd love to get rid of them.

bors · 2019-05-25T10:18:55Z

☀️ Try build successful - checks-travis
Build commit: 01fe51e

RalfJung · 2019-05-25T10:28:36Z

@rust-timer build 01fe51e

rust-timer · 2019-05-25T10:28:38Z

Success: Queued 01fe51e with parent 315ab95, comparison URL.

rust-timer · 2019-05-25T13:04:56Z

Finished benchmarking try commit 01fe51e: comparison url

timvermeulen · 2019-05-25T13:28:58Z

This looks better!

Centril · 2019-05-29T07:22:30Z

r? @scottmcm

jonas-schievink · 2019-06-18T16:11:43Z

Triage: Waiting for review by @rust-lang/libs

scottmcm · 2019-06-20T05:54:34Z

Looks good, thanks @timvermeulen!

@bors r+

bors · 2019-06-20T05:54:36Z

📌 Commit 97a6c93 has been approved by scottmcm

…=scottmcm Implement nth_back for slice::{Iter, IterMut} Part of rust-lang#54054. I implemented `nth_back` as straightforwardly as I could, and then slightly changed `nth` to match `nth_back`. I believe I did so correctly, but please double-check 🙂 I also added the helper methods `zst_shrink`, `next_unchecked`, and `next_back_unchecked` to get rid of some duplicated code. These changes hopefully make this code easier to understand for new contributors like me. I noticed the `is_empty!` and `len!` macros which sole purpose seems to be inlining, according to the comment right above them, but the `is_empty` and `len` methods are already marked with `#[inline(always)]`. Does that mean we could replace these macros with method calls, without affecting anything? I'd love to get rid of them.

RalfJung · 2019-06-20T07:32:12Z

@bors rollup-
due to possible perf impact

RalfJung · 2019-06-20T07:33:04Z

Well that didn't do anything. And it's already been rolled up anyway... whatever^^

@ghost

Rollup of 4 pull requests Successful merges: - #60454 (Add custom nth_back to Skip) - #60772 (Implement nth_back for slice::{Iter, IterMut}) - #61782 (suggest tuple struct syntax) - #61968 (rustc: disallow cloning HIR nodes.) Failed merges: r? @ghost

bors · 2019-06-20T10:40:10Z

⌛ Testing commit 97a6c93 with merge 4fb77a0...

rust-highfive assigned TimNN May 12, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 12, 2019

RalfJung reviewed May 13, 2019

View reviewed changes

Implement nth_back for slice::{Iter, IterMut}

97a6c93

rust-highfive assigned scottmcm and unassigned TimNN May 29, 2019

koalatux mentioned this pull request Jun 8, 2019

Slow performance of std::iter::Rev with iterator adapters using std::iter::Iterator::nth() #54054

Open

25 tasks

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 20, 2019

Centril mentioned this pull request Jun 20, 2019

Rollup of 4 pull requests #61983

Merged

bors merged commit 97a6c93 into rust-lang:master Jun 20, 2019

timvermeulen deleted the slice_iter_nth_back branch June 27, 2019 07:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement nth_back for slice::{Iter, IterMut} #60772

Implement nth_back for slice::{Iter, IterMut} #60772

timvermeulen commented May 12, 2019

rust-highfive commented May 12, 2019

RalfJung May 13, 2019

timvermeulen May 13, 2019

RalfJung May 13, 2019

timvermeulen May 13, 2019

RalfJung May 13, 2019

timvermeulen May 13, 2019

RalfJung commented May 13, 2019

bors commented May 13, 2019

bors commented May 13, 2019

rust-highfive commented May 13, 2019

timvermeulen commented May 13, 2019

RalfJung commented May 13, 2019

bors commented May 13, 2019

bors commented May 13, 2019

RalfJung commented May 14, 2019

rust-timer commented May 14, 2019

timvermeulen commented May 14, 2019 •

edited

Loading

timvermeulen commented May 24, 2019

bors commented May 24, 2019

RalfJung commented May 25, 2019

bors commented May 25, 2019

bors commented May 25, 2019

RalfJung commented May 25, 2019

rust-timer commented May 25, 2019

rust-timer commented May 25, 2019

timvermeulen commented May 25, 2019

Centril commented May 29, 2019

jonas-schievink commented Jun 18, 2019

scottmcm commented Jun 20, 2019

bors commented Jun 20, 2019

RalfJung commented Jun 20, 2019

RalfJung commented Jun 20, 2019

bors commented Jun 20, 2019

Implement nth_back for slice::{Iter, IterMut} #60772

Implement nth_back for slice::{Iter, IterMut} #60772

Conversation

timvermeulen commented May 12, 2019

rust-highfive commented May 12, 2019

RalfJung May 13, 2019

Choose a reason for hiding this comment

timvermeulen May 13, 2019

Choose a reason for hiding this comment

RalfJung May 13, 2019

Choose a reason for hiding this comment

timvermeulen May 13, 2019

Choose a reason for hiding this comment

RalfJung May 13, 2019

Choose a reason for hiding this comment

timvermeulen May 13, 2019

Choose a reason for hiding this comment

RalfJung commented May 13, 2019

bors commented May 13, 2019

bors commented May 13, 2019

rust-highfive commented May 13, 2019

timvermeulen commented May 13, 2019

RalfJung commented May 13, 2019

bors commented May 13, 2019

bors commented May 13, 2019

RalfJung commented May 14, 2019

rust-timer commented May 14, 2019

timvermeulen commented May 14, 2019 • edited Loading

timvermeulen commented May 24, 2019

bors commented May 24, 2019

RalfJung commented May 25, 2019

bors commented May 25, 2019

bors commented May 25, 2019

RalfJung commented May 25, 2019

rust-timer commented May 25, 2019

rust-timer commented May 25, 2019

timvermeulen commented May 25, 2019

Centril commented May 29, 2019

jonas-schievink commented Jun 18, 2019

scottmcm commented Jun 20, 2019

bors commented Jun 20, 2019

RalfJung commented Jun 20, 2019

RalfJung commented Jun 20, 2019

bors commented Jun 20, 2019

timvermeulen commented May 14, 2019 •

edited

Loading