Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display assertion failure immediately once falsified in text mode #1271

Merged
merged 1 commit into from
Jun 13, 2024

Conversation

samalws-tob
Copy link
Collaborator

Fixes #1205

Example:
Before:

Analyzing contract: /Users/sam/Documents/echidna-june-2024/echidna/contract.sol:Foo
[2024-06-12 11:23:17.57] Running slither on contract.sol... Done! (0.702374s)
[2024-06-12 11:23:18.27] [Worker 0] Test echidna_asdf falsified!
[2024-06-12 11:23:18.33] [Worker 0] New coverage: 256 instr, 1 contracts, 1 seqs in corpus
[2024-06-12 11:23:20.83] [Worker 0] Test limit reached. Stopping.
[2024-06-12 11:23:20.83] [status] tests: 1/1, fuzzing: 101/50000, values: [], cov: 256, corpus: 1
echidna_asdf: failed!💥  
  Call sequence:
    Foo.asdf(1)
    Foo.bsdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)

Traces: 



Unique instructions: 256
Unique codehashes: 1
Corpus size: 1
Seed: 3826264006055040584

After:

Analyzing contract: /Users/sam/Documents/echidna-june-2024/echidna/contract.sol:Foo
[2024-06-12 11:23:17.57] Running slither on contract.sol... Done! (0.702374s)
[2024-06-12 11:23:18.27] [Worker 0] Test echidna_asdf falsified!
  Call sequence:
Foo.csdf(57947087071538287116229096099518778219345908371936822082628725600138693436524) from: 0x0000000000000000000000000000000000020000 Time delay: 390247 seconds Block delay: 53678
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639930) from: 0x0000000000000000000000000000000000020000 Time delay: 344203 seconds Block delay: 59552
Foo.bsdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 33605 seconds Block delay: 5053
Foo.csdf(0) from: 0x0000000000000000000000000000000000020000 Time delay: 117472 seconds Block delay: 1362
Foo.bsdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 332369 seconds Block delay: 11942
Foo.asdf(4370000) from: 0x0000000000000000000000000000000000020000 Time delay: 590349 seconds Block delay: 60267
Foo.asdf(3) from: 0x0000000000000000000000000000000000020000 Time delay: 113273 seconds Block delay: 60364
Foo.csdf(105328709578986275286875549616058049166248807697387859268196766548365176826234) from: 0x0000000000000000000000000000000000010000 Time delay: 45142 seconds Block delay: 50499
Foo.csdf(336) from: 0x0000000000000000000000000000000000010000 Time delay: 198598 seconds Block delay: 1984
Foo.bsdf(24495534881220516422514047452236933939549535757623267174267198127854015088056) from: 0x0000000000000000000000000000000000020000 Time delay: 522178 seconds Block delay: 37767
Foo.asdf(973821506116829516598871744363094460124616298305013842383472628649812407579) from: 0x0000000000000000000000000000000000030000 Time delay: 32767 seconds Block delay: 5054
Foo.asdf(62737908824103347051668683296154850097080317044880093722682446698363781686349) from: 0x0000000000000000000000000000000000030000 Time delay: 275394 seconds Block delay: 561
Foo.csdf(2) from: 0x0000000000000000000000000000000000030000 Time delay: 404997 seconds Block delay: 32737
Foo.bsdf(16673238516546302010005614697145054190922750209626982441491245815592674170811) from: 0x0000000000000000000000000000000000010000 Time delay: 73040 seconds Block delay: 23978
Foo.bsdf(0) from: 0x0000000000000000000000000000000000010000 Time delay: 400981 seconds Block delay: 5023
Foo.asdf(4369999) from: 0x0000000000000000000000000000000000020000 Time delay: 67960 seconds Block delay: 5140
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639929) from: 0x0000000000000000000000000000000000030000 Time delay: 254414 seconds Block delay: 32
Foo.csdf(5) from: 0x0000000000000000000000000000000000020000 Time delay: 303345 seconds Block delay: 53562
Foo.bsdf(4369999) from: 0x0000000000000000000000000000000000030000 Time delay: 303345 seconds Block delay: 57123
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639932) from: 0x0000000000000000000000000000000000030000 Time delay: 361136 seconds Block delay: 53562
Foo.bsdf(4370001) from: 0x0000000000000000000000000000000000020000 Time delay: 401699 seconds Block delay: 30011
Foo.asdf(98404317591302697895728486117905658464611205800369243083584335400803065093621) from: 0x0000000000000000000000000000000000020000 Time delay: 405856 seconds Block delay: 55538
Foo.asdf(82181221751846409617278589711486435926481222565914779440220456813086146204352) from: 0x0000000000000000000000000000000000010000 Time delay: 77600 seconds Block delay: 35334
Foo.csdf(79872180235038370578240087930809364320283939460722175098901465838627851118130) from: 0x0000000000000000000000000000000000020000 Time delay: 361136 seconds Block delay: 53562
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639931) from: 0x0000000000000000000000000000000000020000 Time delay: 338920 seconds Block delay: 7323
Foo.bsdf(41661192703382420058606248607496353973014862028674264586095731523835873884449) from: 0x0000000000000000000000000000000000020000 Time delay: 156190 seconds Block delay: 6234
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639929) from: 0x0000000000000000000000000000000000030000 Time delay: 271957 seconds Block delay: 5
Foo.bsdf(7) from: 0x0000000000000000000000000000000000020000 Time delay: 66543 seconds Block delay: 255
Foo.asdf(56460771626007978680176703879577391580640111980800679099119100744772346902825) from: 0x0000000000000000000000000000000000020000 Time delay: 338920 seconds Block delay: 255
Foo.bsdf(4369999) from: 0x0000000000000000000000000000000000020000 Time delay: 4177 seconds Block delay: 5022
Foo.csdf(1524785993) from: 0x0000000000000000000000000000000000020000 Time delay: 297507 seconds Block delay: 36859
Foo.bsdf(91191941997648570331845782433300455060714231629880791567007886618864267748840) from: 0x0000000000000000000000000000000000010000 Time delay: 19029 seconds Block delay: 7323
Foo.asdf(1) from: 0x0000000000000000000000000000000000030000 Time delay: 4177 seconds Block delay: 2
Foo.asdf(15455063137949734296523602836120957796090026793672118834464526419900710503005) from: 0x0000000000000000000000000000000000030000 Time delay: 166184 seconds Block delay: 42229
Foo.csdf(115792089237316195423570985008687907853269984665640564039457584007913129639934) from: 0x0000000000000000000000000000000000020000 Time delay: 358061 seconds Block delay: 4896
Foo.bsdf(966) from: 0x0000000000000000000000000000000000020000 Time delay: 405856 seconds Block delay: 42985
Foo.bsdf(115792089237316195423570985008687907853269984665640564039457584007913129639932) from: 0x0000000000000000000000000000000000020000 Time delay: 295050 seconds Block delay: 53011
Foo.asdf(7) from: 0x0000000000000000000000000000000000020000 Time delay: 434894 seconds Block delay: 53349
Foo.bsdf(52841453647672496968826193256958640800734554231771408205956281745404396507200) from: 0x0000000000000000000000000000000000010000 Time delay: 127 seconds Block delay: 32
Foo.bsdf(42952923681104225761253541795653292684036272771431273924024132718777897343624) from: 0x0000000000000000000000000000000000020000 Time delay: 33271 seconds Block delay: 12155
Foo.csdf(4369999) from: 0x0000000000000000000000000000000000010000 Time delay: 487078 seconds Block delay: 57086

[2024-06-12 11:23:18.33] [Worker 0] New coverage: 256 instr, 1 contracts, 1 seqs in corpus
[2024-06-12 11:23:20.83] [Worker 0] Test limit reached. Stopping.
[2024-06-12 11:23:20.83] [status] tests: 1/1, fuzzing: 101/50000, values: [], cov: 256, corpus: 1
echidna_asdf: failed!💥  
  Call sequence:
    Foo.asdf(1)
    Foo.bsdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)
    Foo.csdf(1)

Traces: 



Unique instructions: 256
Unique codehashes: 1
Corpus size: 1
Seed: 3826264006055040584

@ggrieco-tob
Copy link
Member

Can we signal the user how big the sequence call at each step of the shrinking?

@samalws-tob
Copy link
Collaborator Author

We could do that, how often do you think we should log? If we log every shrinking step we'd end up with thousands of log messages, maybe we could log every 100 steps or every 1000 steps, or every time the size decreases significantly

@samalws-tob samalws-tob marked this pull request as ready for review June 12, 2024 17:41
@ggrieco-tob
Copy link
Member

The shrink log should output something if the size of the sequence of transaction is reduced. Normally, it should not be more than 100 steps (max).

@ggrieco-tob
Copy link
Member

Let me merge this one and we can talk about other small fixes later.

@ggrieco-tob ggrieco-tob merged commit 9f05340 into master Jun 13, 2024
17 of 18 checks passed
@ggrieco-tob ggrieco-tob deleted the showTxsInTextMode branch June 13, 2024 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider displaying assertion failure immediately once falsified in text mode
2 participants