JIT zig fmt and zig reduce

[andrewrk]

Implementation of #19063 for zig fmt and zig reduce.

Benchmark 1 (17 runs): master-zig fmt /home/andy/dev/zig/lib/std
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           299ms ± 2.77ms     291ms …  302ms          3 (18%)        0%
  peak_rss           51.1MB ± 77.9KB    51.0MB … 51.3MB          2 (12%)        0%
  cpu_cycles         1.23G  ±  898K     1.23G  … 1.24G           1 ( 6%)        0%
  instructions       2.93G  ±  689      2.93G  … 2.93G           0 ( 0%)        0%
  cache_references   6.73M  ±  129K     6.53M  … 7.01M           0 ( 0%)        0%
  cache_misses        120K  ± 6.84K      102K  …  134K           4 (24%)        0%
  branch_misses      7.90M  ± 5.38K     7.89M  … 7.92M           1 ( 6%)        0%
Benchmark 2 (17 runs): branch-zig fmt /home/andy/dev/zig/lib/std
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           299ms ± 2.38ms     293ms …  302ms          0 ( 0%)          -  0.2% ±  0.6%
  peak_rss           57.3MB ±    0      57.3MB … 57.3MB          0 ( 0%)        💩+ 12.1% ±  0.1%
  cpu_cycles         1.18G  ±  998K     1.17G  … 1.18G           0 ( 0%)        ⚡-  4.6% ±  0.1%
  instructions       2.66G  ±  716      2.66G  … 2.66G           0 ( 0%)        ⚡-  9.2% ±  0.0%
  cache_references   6.06M  ± 71.2K     5.91M  … 6.16M           0 ( 0%)        ⚡-  9.9% ±  1.1%
  cache_misses        116K  ± 1.46K      114K  …  120K           1 ( 6%)          -  3.6% ±  2.9%
  branch_misses      7.81M  ± 4.31K     7.80M  … 7.81M           0 ( 0%)        ⚡-  1.2% ±  0.0%

The first run takes 10 seconds or so but when every run is a cache hit, the timing difference is insignificant, or somehow faster even.

This makes the rendering code (i.e. lib/std/zig/render.zig) dead for primary compiler builds. This did not have much of an effect on how long it takes to build the compiler (see below) probably because it's just not that much code, and it's not using any slow stuff like gratuitous formatting, comptime blocks, or use of the inline keyword.

Benchmark 1 (3 runs): before/zig build-exe ...
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           102s  ± 4.64s     97.3s  …  107s           0 ( 0%)        0%
  peak_rss           4.60GB ±  220KB    4.60GB … 4.60GB          0 ( 0%)        0%
  cpu_cycles          402G  ± 1.18G      401G  …  403G           0 ( 0%)        0%
  instructions        588G  ±  127M      588G  …  589G           0 ( 0%)        0%
  cache_references   26.2G  ±  103M     26.1G  … 26.3G           0 ( 0%)        0%
  cache_misses       3.05G  ± 14.2M     3.04G  … 3.07G           0 ( 0%)        0%
  branch_misses      2.53G  ± 1.73M     2.53G  … 2.53G           0 ( 0%)        0%
Benchmark 2 (3 runs): after/zig build-exe ...
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           104s  ± 1.93s      102s  …  106s           0 ( 0%)          +  1.7% ±  7.9%
  peak_rss           4.64GB ± 30.3MB    4.62GB … 4.68GB          0 ( 0%)          +  0.9% ±  1.1%
  cpu_cycles          397G  ± 7.71G      392G  …  406G           0 ( 0%)          -  1.3% ±  3.1%
  instructions        580G  ± 12.3G      572G  …  594G           0 ( 0%)          -  1.5% ±  3.4%
  cache_references   23.3G  ±  340M     23.1G  … 23.7G           0 ( 0%)        ⚡- 11.3% ±  2.2%
  cache_misses       3.05G  ± 11.6M     3.04G  … 3.06G           0 ( 0%)          -  0.0% ±  1.0%
  branch_misses      2.56G  ± 55.8M     2.53G  … 2.63G           0 ( 0%)          +  1.5% ±  3.5%

I expect to see more time saved when applying this to Aro, Resinator, translate-c, etc., which represent a much more substantial amount of code.

After this branch is merged, it becomes easier to contribute to zig fmt and zig reduce. Contributors no longer need to rebuild the compiler to test changes. You can use ZIG_DEBUG_CMD=1 zig fmt ... to get a debug build of the format subcommand, which could be useful when working on zig fmt itself. Same thing for zig reduce.

[nektro]

very exciting! one suggestion I'd make is moving fmt's main to tools/ or a new bins/ folder so that userspace code isn't accidentally able to call std.zig.fmt.main().

[FitsumKurabachew]

ችኅን