Slow editor startup with large number of opened scripts in the script editor

[Xrayez]

Godot version:
3.2.2.stable, 4.0-dev

OS/device including version:
Windows 10

Issue description:
Starting a project with over 100 scripts opened in the script editor negatively affects the startup times.

On a project with 500 small scripts opened in the script editor, it takes around 15-20 seconds before the editor becomes responsive. If you close all scripts with File → Close All and restart Godot, it takes around 3-6 seconds, as usual.

Steps to reproduce:

1. Open any project with over 100 scripts.
2. Open each individual script so that it becomes listed in the script editor.
3. Restart Godot.

It's quite time consuming to do this manually, so proceed to the minimal project instructions.

Minimal reproduction project:
script-editor-slow-startup.zip (includes 500 scripts)

1. Unpack minimal project.

2. Run the project.

3. Close the project.

4. Copy and replace the included editor_layout.cfg file to the path where project-specific editor settings and cache are stored (on Windows the path may be similar to C:\Users\Profile\AppData\Roaming\Godot\projects\script-editor-slow-startup-%MD5%). That way when you open Godot, all the project scripts will be opened in the script editor. You can also navigate this via menu:

   

5. Run the project and notice slow startup.

Notes:

1. MessageQueue size is increased to allow loading over 500 scripts in the minimal project (Change MessageQueue to a page allocator to prevent overflow #35653).
2. Panel dragging (split container) becomes very laggy. The window layout info is constantly written to editor_layout.cfg while doing so, and it seems like it does this for each and every script as well.
3. Close All produces similar editor hang as the editor startup.

Workaround:
There's currently no editor option to disable this feature. Make sure to close old opened scripts regularly. Actually there is: #39841 (comment).
I've created a plugin which does that automatically, bypassing this issue currently in 3.2: #39841 (comment).

Related issues
Marginally related to #9815.

[Calinou]

Panel dragging (split container) becomes very laggy. The window layout info is constantly written to editor_layout.cfg while doing so, and it seems like it does this for each and every script as well.

It should probably save layouts only when the editor closes or when asked to do so using Save Layout.

[Xrayez]

Yeah that's probably one of the problems to address.

I tested this on master (87d2397 4.0-dev) with target=debug. Takes like whopping 8 minutes (likely much less with target=release_debug) to open the minimal project with 500 scripts opened in the script editor. 🥇

During this time I've had an opportunity to capture the real-time I/O shenanigans Godot does with the project:

As you see, it's not only the layout, but project_metadata.cfg read/write.

When it finally stopped, it crashed:

CrashHandlerException: Program crashed
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[0] vk_optimusGetInstanceProcAddr
[1] vkGetInstanceProcAddr
[3] RenderingDeviceVulkan::prepare_screen_for_drawing (drivers\vulkan\rendering_device_vulkan.cpp:6873)
[4] RasterizerRD::prepare_for_blitting_render_targets (servers\rendering\rasterizer_rd\rasterizer_rd.cpp:37)
[5] RenderingServerViewport::draw_viewports (servers\rendering\rendering_server_viewport.cpp:454)
[6] RenderingServerRaster::draw (servers\rendering\rendering_server_raster.cpp:113)
[7] RenderingServerWrapMT::draw (servers\rendering\rendering_server_wrap_mt.cpp:93)
[8] Main::iteration (main\main.cpp:2218)
[9] OS_Windows::run (platform\windows\os_windows.cpp:627)
[10] widechar_main (platform\windows\godot_windows.cpp:161)
[11] _main (platform\windows\godot_windows.cpp:183)
[12] main (platform\windows\godot_windows.cpp:195)
[13] __scrt_common_main_seh (d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[14] BaseThreadInitThunkrocAddr
[1] vkGetInstanceProcAddr
[3] RenderingDeviceVulkan::prepare_screen_for_drawing (D:\src\godot\drivers\vulkan\rendering_device_vulkan.cpp:6873)
[4] RasterizerRD::prepare_for_blitting_render_targets (D:\src\godot\servers\rendering\rasterizer_rd\rasterizer_rd.cpp:37)
[5] RenderingServerViewport::draw_viewports (D:\src\godot\servers\rendering\rendering_server_viewport.cpp:454)
[6] RenderingServerRaster::draw (D:\src\godot\servers\rendering\rendering_server_raster.cpp:113)
[7] RenderingServerWrapMT::draw (D:\src\godot\servers\rendering\rendering_server_wrap_mt.cpp:93)
[8] Main::iteration (D:\src\godot\main\main.cpp:2218)
[9] OS_Windows::run (D:\src\godot\platform\windows\os_windows.cpp:627)
[10] widechar_main (D:\src\godot\platform\windows\godot_windows.cpp:161)
[11] _main (D:\src\godot\platform\windows\godot_windows.cpp:183)
[12] main (D:\src\godot\platform\windows\godot_windows.cpp:195)
[13] __scrt_common_main_seh (d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[14] BaseThreadInitThunk

Not related to this issue, because I've got these kind of crashes previously on master, when you focus out and in Godot editor it seems.

I may take a look at solving this on 3.2 branch (cannot use master due to crashes as above), but feel free to investigate as well because I might not be able to solve this by myself.

[Paulb23]

See #33760

[Xrayez]

Thanks, surprisingly I'm actually the one who reacted with 🎉 on that PR months ago, but I haven't experienced the issue back then to realize the performance impact, perfect timing though. 😄

[Xrayez]

Actually I've created a small plugin which allows you to automatically clear the list of opened scripts on editor exit (I don't see how seeing the history from other sessions is useful personally). This alone workarounds and shadows the underlying issue, so this can be closed. 😛

But this is still an issue nonetheless, and I wonder whether #33760 could provide other run-time performance enhancements other than startup times, and likely yes due to:

Add to "recently opened" now only when opening a new script (causes load/save metadata file)

which is characterized by quite laggy panel split dragging as described in the issue.

[Xrayez]

Actually I've figured there's already an editor setting for that...

text_editor/files/restore_scripts_on_load = false

Expected it to be under text_editor/script_list. 😃

So the above plugin as a workaround is overkill indeed, unless you want to prevent documentation pages to be there specifically.