Profiling studies on more platforms

[wlandau]

Prework

• Read and abide by drake's code of conduct.
• Search for duplicates among the existing issues, both open and closed.
• Advanced users: verify that the bottleneck still persists in the current development version (i.e. remotes::install_github("ropensci/drake")) and mention the SHA-1 hash of the Git commit you install.

Description

drake is much faster than it used to be, but it always needs work. I have profiled a bunch on my home Linux machine and to some degree on the Mac and Linux machines I can access at work. We may unearth new bottlenecks if we run profiling studies on Windows and on rigs with slow file systems.

You can help

I would really appreciate your help! The easiest way to contribute is with https://github.com/ropensci/drake/blob/master/.github/ISSUE_TEMPLATE/bottleneck.md#benchmarks, and I have an existing profiling workflow here. static.R is a pretty good benchmark, though if your system is not super powerful, maybe attenuate seq_len(1e4) in the plan to something smaller. The instructions generalize well, and if you want to plug in your own plan, it would really help us cover a more diverse set of use cases.

[wlandau]

I forgot to emphasize: the flame graphs from pprof are particularly helpful ( example here). In an ideal world, we would use profvis because it is easier to install and use, but right now it runs into performance issues pretty quickly (r-lib/profvis#87 and r-lib/profvis#104).

[wlandau]

I just did some profiling on a personal ThinkPad X1 Carbon (6th Gen) with these specs:

• Intel Core i5-8250U MB
• 8GB LPDDRhttps://github.com/wlandau/drake-examples/blob/master/overhead/setup.R3 2133 MB
• 512GB SSD M.2 2280 NVMe OPAL2
• Kubuntu 18.04 and Windows 10 (dual boot)

On Linux, results were similar to what I have been seeing on other machines.

On the Windows partition of the same machine, interaction with the file system was much slower. Here are profiling results for the same workflow as before.

This reminds me of #937. Some machines just have slower file systems, and that is going to matter. I think we are already saving targets as fast as possible, especially with custom data formats. But maybe there is something we can do about all the tiny file operations that do bookkeeping: metadata, progress, history, and data recovery. Perhaps those should exist outside storr. Maybe we can use memory-mapped files. I do not know.

Related: #937.

[wlandau]

Looking at the flame graph on Windows, it looks like we could gain some speed with a faster alternative to file.rename().

[wlandau]

I think we're getting the word out with #1086 and #1089.