Polygonal video re-creation

[insomniacUNDERSCORElemon]

Context

There's an AI tool called ebsynth that allows copying the style in a few frames over the entire video. For instance see Joel Haver's ebynth-assisted animations/rotoscoping on Youtube, specifically the video titled How I Animated This Video where he shows the steps and details.

Using Godot to make input frames might actually work if you just want the look, but ebsynth puts out very wobbly frames from what I've seen, so I don't think it'd work well particularly for more minimalist stuff (especially flat edges and sharp corners).

Idea

A similar thing to ebsynth-assisted rotoscoping, but inside Godot, specifically with polygon2d and line2d nodes. Because working with vector should give greater flexibility in both production and results. With ebsynth when you find issues you'd make an additional style frame and re-render, with this you could edit the problem frame directly.

Aside from the benefit of scalability that ebsynth cannot offer (you'd have to re-render your video at a higher res), the benefit of being in Godot directly would be crisp animations that have a very low data footprint (you wouldn't need to have the input video in your exported project).

It might also be faster than working with video output, and might give more-intended results at least in the case of minimalist style (less data to work with).

Potential issues

The biggest issue I see is that polygonal editors in Godot still aren't great (no vertex color painting, no multi-vertex selection etc). Although AI assistance does mitigate that as you'd be doing significantly less work than if doing it all manually.

It will likely be an issue to create interesting/complex designs that any sort of AI would know how to manipulate correctly. The 4.0 option clip children (which is currently broken, see my issue 49198 on the Godot issue tracker) might make this easier, as transforming nodes will likely be easier than doing something like knowing what to do with vertex colors (at least w/shading). If it could use the tree structure and naming used (particularly for things like eyes and mouth) that might help things out. It might also be useful to control how certain nodes are manipulated, for instance restricting certain nodes to not have their vertices edited directly (instead only skew/scale/rotation/moving) particularly if you can also use this to add additional data (like if you could make character with a pseudo-3D cube head).