WIP: Parallelize passes using rayon

[Zoxc]

This builds on #46193 and #45912 and actually makes code run in parallel. This is not quite ready yet since rustc is not yet completely thread safe. It also uses a rough fork of rayon which uses fibers/stackful coroutines.

[Zoxc]

@bors try

[bors]

⌛ Trying commit c3e21ab7c517fd303077036df3b211cd28bf7b0e with merge 4f1b9ac80d435420ca47d60344b959db6e8e4f51...

[bors]

💔 Test failed - status-travis

[bors]

☔ The latest upstream changes (presumably #46509) made this pull request unmergeable. Please resolve the merge conflicts.

[Zoxc]

@bors try

[bors]

⌛ Trying commit 086a5fb with merge 463435e9a0879e3db40c9ed64f12a146f4937b16...

[bors]

💔 Test failed - status-travis

[Zoxc]

@bors try

[bors]

⌛ Trying commit 26a0515 with merge 8ad7c34...

[Zoxc]

@Mark-Simulacrum I'd also like a perf run here when the try build is complete

[bors]

☀️ Test successful - status-travis
State: approved= try=True

[Mark-Simulacrum]

Perf run queued.

[kennytm]

Perf result FYI: http://perf.rust-lang.org/compare.html?start=2537a499c2f1983011ad04ba2e59e5ff2e2989b6&end=8ad7c343c0a178cc8cf419f4fe41cf46c0bc84f4&stat=wall-time

[mark-i-m]

What are the specs of the machine on which this is running? Does it have the cores to actually take advantage of this PR?

[Mark-Simulacrum]

In general, yes, it should. I don't have the exact specs right now, though -- it has ~8 cores IIRC.

[Zoxc]

@Mark-Simulacrum I'd also like to know the specs of the machine and also I'd like to know how it is configured; does the CPU run with constant frequency? are there any background services running?

Note that the try build is with thread safety and the rayon thread pool disabled.

[mark-i-m]

Note that the try build is with thread safety and the rayon thread pool disabled.

Ah, that makes more sense... We were measuring the overhead of thread-safety.

[Zoxc]

Lets try that again.

Note that the try build is with (thread safety and the rayon thread pool) disabled.

It is just measuring what the cost of merging this with everything disabled is.

[bors]

☔ The latest upstream changes (presumably #46335) made this pull request unmergeable. Please resolve the merge conflicts.

[rust-highfive]

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

[bors]

☔ The latest upstream changes (presumably #50699) made this pull request unmergeable. Please resolve the merge conflicts.

[mrhota]

@Zoxc what's the story with this PR? is it dead?

[Zoxc]

An alternative approach which doesn't use fibers is merged in master. This still parallelizes more code though. That should be cherry picked out and analyzed (it may not be beneficial to parallelize all passes).

[mikumikuchan]

sorry I mis clicked "approve" and I don't know why some how it succeeded. please discard my approval.

[mark-i-m]

@mikumikuchan No worries! That's the Github code review feature. The rust project instead uses @bors commands, which have an ACL (which I'm not on :P ).

[TimNN]

Ping from triage, @Zoxc: What is the status of this?

[TimNN]

Ping from triage, @Zoxc: we haven't heard from you for a while, so we are closing this PR. Feel free to re-open it in the future. Thanks for your contribution!