Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Readd shallow variables heuristic #621

Merged
merged 1 commit into from
Feb 29, 2024

Conversation

SSoelvsten
Copy link
Owner

@SSoelvsten SSoelvsten commented Feb 27, 2024

Add the "shallow variables" heuristic anew to revert the slowdown introduced in #608 .

@SSoelvsten SSoelvsten self-assigned this Feb 27, 2024
@SSoelvsten SSoelvsten added the ✨ optimisation It's all about speed / space label Feb 27, 2024
Copy link

codecov bot commented Feb 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.232%. Comparing base (fc67136) to head (c98eeb2).

Additional details and impacted files
@@              Coverage Diff              @@
##              main      #621       +/-   ##
=============================================
+ Coverage   97.225%   97.232%   +0.007%     
=============================================
  Files           90        90               
  Lines         6343      6358       +15     
=============================================
+ Hits          6167      6182       +15     
  Misses         176       176               

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Feb 27, 2024

🟡 Regression Test (QBF 'breakthrough/3x4_19_bwnib')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of -0.07% (stdev: 1.76%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 24798.50 24815.25
Standard Deviation 254.11 437.71

Number of samples: 4

Copy link

github-actions bot commented Feb 27, 2024

🟢 Regression Test (Picotrav 'mem_ctrl')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of 1.37% (stdev: 2.64%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 116333.33 114740.67
Standard Deviation 3076.43 2836.94

Number of samples: 3

Copy link

github-actions bot commented Feb 27, 2024

🟢 Regression Test (QBF 'breakthrough/3x5_11_bwnib')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of 1.22% (stdev: 1.63%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 7505.40 7414.20
Standard Deviation 122.41 56.55

Number of samples: 5

Copy link

github-actions bot commented Feb 27, 2024

🟡 Regression Test (Picotrav 'arbiter')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of -0.36% (stdev: 1.28%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 62847.50 63075.25
Standard Deviation 807.07 343.97

Number of samples: 4

Copy link

github-actions bot commented Feb 27, 2024

🟡 Regression Test (QBF 'domineering/5x5_13_bwnib')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of -2.44% (stdev: 4.65%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 22471.17 23019.83
Standard Deviation 1046.03 120.64

Number of samples: 6

Copy link

github-actions bot commented Feb 27, 2024

🟡 Regression Test (Picotrav 'adder')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of -0.03% (stdev: 0.63%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 9166.40 9169.20
Standard Deviation 57.53 44.20

Number of samples: 5

Copy link

github-actions bot commented Feb 27, 2024

🟢 Regression Test (QBF 'ep/8x8_7_e-8-1_p-3-4_bwnib')

'origin/internal/quantify/shallow_variables_heuristic' is a change in performance of 59.51% (stdev: 8.60%).

... origin/main origin/internal/quantify/shallow_variables_heuristic
Mean 28429.60 11512.20
Standard Deviation 2444.69 81.77

Number of samples: 10

@SSoelvsten
Copy link
Owner Author

SSoelvsten commented Feb 27, 2024

As can be seen from the above, this heuristic reverts the slowdown in #608 for ep/8x8. Now, the unit tests just need to be updated accordingly to make this mergeable.

@SSoelvsten SSoelvsten force-pushed the internal/quantify/shallow_variables_heuristic branch 3 times, most recently from 4df091f to 8c4971a Compare February 29, 2024 08:19
@SSoelvsten SSoelvsten force-pushed the internal/quantify/shallow_variables_heuristic branch from 8c4971a to c98eeb2 Compare February 29, 2024 08:49
@SSoelvsten SSoelvsten marked this pull request as ready for review February 29, 2024 08:49
@SSoelvsten
Copy link
Owner Author

Interestingly this does not seem to fully fix domineering/5x5_13_bwnib. So, we may also want to readd the heuristic based on the number of terminal arcs (or an improved version of it).

@SSoelvsten SSoelvsten merged commit 07c6dda into main Feb 29, 2024
56 checks passed
@SSoelvsten SSoelvsten deleted the internal/quantify/shallow_variables_heuristic branch February 29, 2024 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ optimisation It's all about speed / space
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant