Fix crashes due to stack overflow when painting a large area in tile map

[komugi1211s]

Fixes #76473.

This PR takes alloca out of the loop, reuses previous allocations whenever possible to reduce the chance of using up too much stack space.

Cause
the crash described in linked issue is most likely (almost certain) caused by the stack overflow.

filling a large area of tiles at once will create a long list of operations for Undo/Redo to record, which gets processed by the for loop inside UndoRedo::_process_operation_list function, which uses alloca inside. memory obtained from alloca gets freed after you leave the function scope.
given long enough list and right condition (which can be achieved by using a CTRL+SHIFT fill on tile map), alloca can exhaust the stack and cause a stack overflow, resulting in crash.

Valgrind report

each call allocates around 40 bytes (8 bytes for pointer * 5 arguments), resulting in roughly 6.6 MB before the crash

[bitsawer]

I wonder if it would be acceptable to just use LocalVector here with a preallocation call to .resize() (or .reserve()) instead of alloca()? Would result in simpler and more solid code at the expense of one memory allocation+deallocation.

[komugi1211s]

@bitsawer sorry for the late reply -- I think that's a good idea. I kept alloca simply because that is what this code had, replacing with LocalVector makes much more sense.
The performance difference was indistinguishable from one another on my machine as well.

[Calinou]

Tested locally (rebased against master 33957ae) on Linux, it works.

Please squash the commits together so we can merge this. See PR workflow for instructions 🙂

[akien-mga]

Thanks!