chore: Add Instruction::MakeArray to SSA

[jfecher]

Description

Problem*

Resolves #1733

Summary*

Adds a MakeArray instruction to explicitly create a new array. Now no other ValueIds can contain other ValueIds which simplifies a couple passes.

Additional Context

Still working on getting this working in brillig.

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[TomAFrench]

Seems like --show-ssa fixes the compilation failures on the few programs I tried.

[TomAFrench]

Disabling the normalization of valueids shows that we're deleting instructions in the DIE pass which shouldn't be removed. For example the load instruction when returning arr in quicksort is removed from array_sort.

This is happening because somehow the return value of this function (the loaded value) becomes a Value::Function rather than a Value::Instruction. We only mark return values as used if they're constants, instructions or block params so we end up removing the loaded instruction.

[TomAFrench]

I've merged in master so that this branch is up to date so will be handing this PR back now.

[github-actions]

Changes to Brillig bytecode sizes

Generated at commit: 262b9f666c60b916f1c9f82004d6cd04376af4c6, compared to commit: 1cad7c887893ebfb5de57a71d7965c8b88158a14

🧾 Summary (10% most significant diffs)

Program	Brillig opcodes (+/-)	%
databus_two_calldata	-47 ✅	-17.41%
databus_composite_calldata	-151 ✅	-27.61%
databus	-28 ✅	-29.47%
databus_two_calldata_simple	-36 ✅	-30.77%

---

Full diff report 👇

Program	Brillig opcodes (+/-)	%
brillig_unitialised_arrays	48 (+1)	+2.13%
slice_dynamic_index	2,808 (+8)	+0.29%
hashmap	27,594 (-8)	-0.03%
uhashmap	16,716 (-8)	-0.05%
eddsa	11,120 (-15)	-0.13%
brillig_slices	715 (-1)	-0.14%
regression_5252	4,892 (-9)	-0.18%
poseidon_bn254_hash_width_3	5,684 (-12)	-0.21%
poseidon_bn254_hash	5,684 (-12)	-0.21%
slices	2,318 (-5)	-0.22%
poseidonsponge_x5_254	4,494 (-12)	-0.27%
tuple_inputs	375 (-2)	-0.53%
sha2_byte	3,388 (-22)	-0.65%
regression	744 (-5)	-0.67%
nested_array_in_slice	1,279 (-10)	-0.78%
aes128_encrypt	522 (-5)	-0.95%
references	311 (-4)	-1.27%
nested_array_dynamic	2,145 (-36)	-1.65%
bigint	2,045 (-45)	-2.15%
binary_operator_overloading	386 (-9)	-2.28%
conditional_regression_661	127 (-3)	-2.31%
break_and_continue	56 (-5)	-8.20%
regression_3607	44 (-5)	-10.20%
databus_two_calldata	223 (-47)	-17.41%
databus_composite_calldata	396 (-151)	-27.61%
databus	67 (-28)	-29.47%
databus_two_calldata_simple	81 (-36)	-30.77%

[github-actions]

Changes to circuit sizes

Generated at commit: 262b9f666c60b916f1c9f82004d6cd04376af4c6, compared to commit: 1cad7c887893ebfb5de57a71d7965c8b88158a14

🧾 Summary (10% most significant diffs)

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
hashmap	-15,782 ✅	-37.95%	-41,817 ✅	-39.17%

---

Full diff report 👇

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
hashmap	25,806 (-15,782)	-37.95%	64,940 (-41,817)	-39.17%

[jfecher]

This is passing all tests now and showing a surprisingly large improvement in the hashmap test but also large regressions in brillig bytecode sizes.

[TomAFrench]

This is likely due to the fact that constant arrays are no long considered as "constants" by constant_allocation.rs. We're then not hoisting their usage up to avoid multiple initialisations.

[jfecher]

@TomAFrench hmm I tried updating is_constant_value there but doing so just leads to many panics on ValueIds not being available.

[michaeljklein]

LGTM

[jfecher]

Going to hold off on merging this for a bit due to the brillig size regressions

[vezenovm]

Going to hold off on merging this for a bit due to the brillig size regressions

Yeah would be good to avoid this regression.

[sirasistant]

This is likely due to the fact that constant arrays are no long considered as "constants" by constant_allocation.rs. We're then not hoisting their usage up to avoid multiple initialisations.

yes, I think that's the issue. For example, in slices, we have patterns like this:

    inc_rc v207
    v208 = make_slice [Field 0, Field 0]
    inc_rc v208
    v209 = eq v0, v1
    v210 = not v209
    jmpif v210 then: b18, else: b17
  b18():
    v212 = eq v0, Field 20
    v213 = not v212
    jmpif v213 then: b20, else: b19
  b20():
    v214 = make_slice [Field 0, Field 0, v1]
    v215 = make_slice [Field 0, Field 0, v1]
    v216 = make_slice [Field 0, Field 0, v1]
    jmp b21(u32 3, v216)
  b21(v10: u32, v11: [Field]):
    jmp b22(v10, v11)
  b22(v12: u32, v13: [Field]):
    inc_rc v13
    v217 = eq v12, u32 3
    constrain v12 == u32 3
    v218 = lt u32 2, v12
    constrain v218 == u1 1 '"Index out of bounds"'
    v219 = array_get v13, index u32 2
    v220 = eq v219, Field 10
    constrain v219 == Field 10
    v221 = make_slice [Field 0, Field 0]

Previously, these immediate slices would have been picked up by constant_allocation and hoisted to just 1 single initialization. Now, since array constant initialization has been promoted to a instruction, we actually need to do at the ssa level what constant allocation was doing outside SSA. So in order to avoid any regression we need to identify payloads to make_array that are equal to each other and hoist the make_array to the common dominator of all the arrays that are equal.

Also, if all the values are immediates constant_allocation was hoisting them out of loops to avoid extra runtime cost. But that won't affect bytecode size

[jfecher]

fold_2_to_17 -130,344,860 ✅ -99.15%
bench_2_to_17 -144,084,489 ✅ -99.58%

How about this for brillig improvements 😅

[jfecher]

Unfortunately the actual brillig bytecode size is still a bit bigger and most brillig programs still execute more opcodes 🤔

Edit: some of them, like brillig_slices may be because of the lack of the array hoisting optimization. The array deduplication check in constant folding doesn't simplify these since they're all in else branches so there's no single case of these that is in a black which dominates all the others. Fixing this seems somewhat difficult since it'd require checking every element of the array, checking which blocks they were defined in, and trying to hoist to the earliest one. Although doing so could potentially lead to more work if those else blocks were never executed and we hoist out of the if.

[sirasistant]

Although doing so could potentially lead to more work if those else blocks were never executed and we hoist out of the if

Maybe we could hoist out of loops? that one should always lead to better perf I think

[jfecher]

Ended up merging this with the smaller version of the "move instruction to common dominator block" change since the larger, recursive version was failing a few programs.

The diffs now are generally more positive. Executed opcodes can be better or worse depending on the test.