Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarking framework using asv #1120

Merged
merged 2 commits into from
Sep 20, 2022
Merged

Add benchmarking framework using asv #1120

merged 2 commits into from
Sep 20, 2022

Conversation

ianthomas23
Copy link
Member

This PR adds a benchmarking framework to datashader using asv. There are two reasons for doing this. Firstly it allows us to quantify the performance differences within any particular commit of different values of parameters such as antialiased line_width and canvas size. Secondly it allows us to monitor the impact on performance of code changes over time, whether those code changes are intended to improve performance or not.

There is a README file in the benchmarks directory which explains how to use it. So far the benchmarks included are canvas.line for LinesAxis1XConstant, and shade both for categorical and non-categorical data. It is expected that more benchmarks will be added in due course.

Results on my dev machine today are as follows:

$ asv show master
Commit: f1fbae82 <master>

line.Line.time_LinesAxis1XConstant [onion/virtualenv-py3.8-pyct]
  2/12 failed
  ============ =========== ============ ============ ============ ============ ============
  --                                   line_width / self_intersect                         
  ------------ ----------------------------------------------------------------------------
   line_count   0 / False    0 / True    1 / False     1 / True    2 / False     2 / True  
  ============ =========== ============ ============ ============ ============ ============
      1000        failed    6.33±0.1ms    105±20ms    30.3±0.1ms    111±20ms    40.5±0.6ms 
     10000        failed    51.0±0.6ms   1.26±0.01s   273±0.5ms    1.38±0.01s    379±2ms   
  ============ =========== ============ ============ ============ ============ ============
  started: 2022-09-12 14:25:58, duration: 46.6s

shade.Shade.time_shade [onion/virtualenv-py3.8-pyct]
  ok
  ============= ============= ============ ============
  --                              how                  
  ------------- ---------------------------------------
   canvas_size      linear        log        eq_hist   
  ============= ============= ============ ============
       300       10.9±0.04ms   11.4±0.2ms   20.8±0.2ms 
       1000       126±0.6ms    134±0.3ms     231±2ms   
  ============= ============= ============ ============
  started: 2022-09-12 14:26:45, duration: 3.77s

shade.ShadeCategorical.time_shade_categorical [onion/virtualenv-py3.8-pyct]
  ok
  ============= ========= ============ ============ ============ ============
  --                                         category_count                  
  ----------------------- ---------------------------------------------------
   canvas_size     how         3            10           30          100     
  ============= ========= ============ ============ ============ ============
       300        linear   16.6±0.1ms   27.5±0.5ms   58.2±0.7ms    181±2ms   
       300         log     17.6±0.3ms   28.6±0.5ms   59.6±0.6ms   182±0.7ms  
       300       eq_hist   26.9±0.1ms   36.9±0.5ms   67.2±0.3ms   191±0.6ms  
       1000       linear   204±0.6ms    352±0.9ms     730±3ms     2.00±0.01s 
       1000        log     211±0.6ms     361±1ms      740±2ms     2.01±0.01s 
       1000      eq_hist    307±1ms      446±2ms      821±4ms     2.09±0.01s 
  ============= ========= ============ ============ ============ ============
  started: 2022-09-12 14:26:48, duration: 1.21m

Note that the two failed benchmarks are aborted on purpose as non-antialiased lines ignore the self_intersect keyword argument.

@ianthomas23 ianthomas23 merged commit dd01e3e into holoviz:master Sep 20, 2022
@ianthomas23 ianthomas23 deleted the benchmarking branch September 20, 2022 12:01
@ianthomas23 ianthomas23 added this to the v0.14.3 milestone Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant