Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow down and higher memory usage with 2023-04-30 rust toolchain #2457

Closed
zhassan-aws opened this issue May 19, 2023 · 2 comments · Fixed by #2512
Closed

Slow down and higher memory usage with 2023-04-30 rust toolchain #2457

zhassan-aws opened this issue May 19, 2023 · 2 comments · Fixed by #2512
Labels
[C] Bug This is a bug. Something isn't working. [E] Performance Track performance improvement (Time / Memory / CPU) T-CBMC Issue related to an existing CBMC issue

Comments

@zhassan-aws
Copy link
Contributor

With the 2023-04-30 rust toolchain, performance drops significantly on the BTreeSet performance regressions.

For example, with the current main (08659d3), the verification time for tests/perf/btreeset/insert_any is 1.5 seconds:

SUMMARY:
 ** 0 of 1301 failed (60 unreachable)

VERIFICATION:- SUCCESSFUL
Verification Time: 1.4937471s

With #2456, the verification time goes up to 141 seconds:

SUMMARY:
 ** 0 of 1306 failed (60 unreachable)

VERIFICATION:- SUCCESSFUL
Verification Time: 141.20958s

For tests/perf/btreeset/insert_multi, the peak memory usage with main is ~600 MB and verification time is 10 seconds:

SUMMARY:
 ** 0 of 1241 failed (46 unreachable)

 ** 2 of 2 cover properties satisfied


VERIFICATION:- SUCCESSFUL
Verification Time: 10.019767s

With #2456, CBMC runs out of memory (reaches ~26 GB after about 16 minutes).

Similar behavior is observed for tests/perf/btreeset/insert_same.

@zhassan-aws zhassan-aws added [C] Bug This is a bug. Something isn't working. [E] Performance Track performance improvement (Time / Memory / CPU) labels May 19, 2023
@zhassan-aws zhassan-aws changed the title Slow down and more memory usage with 2023-04-30 rust toolchain Slow down and higher memory usage with 2023-04-30 rust toolchain May 19, 2023
@zhassan-aws
Copy link
Contributor Author

These are some of the CBMC statistics on the insert_any example:
image

@tautschnig
Copy link
Member

It seems that the updated toolchain does even more niche optimisation, turning a further tagged union into a niche-optimised one (with the two union members using slightly different member ordering so that a pointer can be used as discriminant (when set to NULL)). diffblue/cbmc#7739 enables CBMC to cope with this case, making sure that field sensitivity can kick in to resolve a comparison against NULL at symbolic execution time.

@tautschnig tautschnig added the T-CBMC Issue related to an existing CBMC issue label May 30, 2023
@zhassan-aws zhassan-aws mentioned this issue Jun 9, 2023
4 tasks
@celinval celinval linked a pull request Jun 9, 2023 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[C] Bug This is a bug. Something isn't working. [E] Performance Track performance improvement (Time / Memory / CPU) T-CBMC Issue related to an existing CBMC issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants