Gas snapshotting wishlist

[transmissions11]

• Account for gas refunds! OG tools does not right now.

• Don't include test call overhead in gas snapshots. OG tools includes some of the overhead rn:

• - --diff flag that shows how much gas usage changed from the last snapshot

  • Summarizes % change of aggregated gas usage across all test funcs
  • Leaderboard for which tests' gas usage changed the most (by %).
    • Show gas units diff and % diff

• - Gas snapshots for fuzz tests

  • Median & mean
  • Gas usage per range of inputs like:

testAddThatExitsEarlyOnLargeNumbers(100-500) = ~5000 gas
testAddThatExitsEarlyOnLargeNumbers(500+) = ~100 gas

• Breakdown of not only gas usage of tests but gas usage of functions the tests call like hardhat-gas-reporter

related: dapphub/dapptools#761

• flamegraphs like tenderly

[lucas-manuel]

Whats left on this? Table formatting?

[wilsoncusack]

I don't think any of this is done except gas for fuzzed tests?

[mds1]

Breakdown of not only gas usage of tests but gas usage of functions the tests call like hardhat-gas-reporter

This one is arguably implemented as you can see gas usage by function in traces, as in #192 (comment), but a report/snapshots showing this would be nice to have

Also I think flamegraphs should be added to this list, below is Tenderly's for reference

@transmissions11 might be worth adding some checkboxes and editing it to link out to PRs once implemented. Mean and median gas on fuzz tests was added in #236

[transmissions11]

@transmissions11 might be worth adding some checkboxes and editing it to link out to PRs once implemented. Mean and median gas on fuzz tests was added in #236

good idea, done. also added flamegraphs

[onbjerg]

Backref #1795

[OliverNChalk]

Just wanted to sanity check the forge snapshot --diff feature. Here is what I currently use, it's:

forge snapshot && git diff --word-diff .gas-snapshot

Here's what forge snapshot --diff produces:

It looks like the gas diff per case is computed as: case_change / total_gas_previous which seems off, I would have expected it to be: case_change / case_previous.

I would also argue that showing before, after, delta is helpful but that may just be personal preference

[wdv4758h]

Is there a place to show Deployment Cost diff right now?
Not sure if this is in the snapshot scope, but I will be happy if it's in snapshot diff.
I've been working to cut the contract size, tracking the size in snapshot helps.

[lucas-manuel]

Close-able? @mds1

[mds1]

Closing in favor of #2056