Project manager scan should run in a thread to keep responsive UI

[Biarity]

Operating system or device - Godot version:
Windows 10, Godot 2.1rc1

Issue description (what happened, and what was expected):
Using the scan function at the project manager crashes Godot when I click 'open'.

Steps to reproduce:
Open Godot, click Scan, select a large folder, click Open. Shows the Windows Close/Debug error box.

[vnen]

I cannot reproduce this. The project manager stops responding while it is searching for files but if I leave it running it manages to fill the list.

Maybe moving the search to a separate thread might be good so the editor keeps responsive while searching.

[Biarity]

Try one of the folders in C:\Users on Windows. Other folders on my Desktop seem to work fine.

[vnen]

For me it just hangs for a long time scanning the folders. On the Downloads folder (which has just a few subfolders) it finishes pretty much instantly.

[bojidar-bg]

Any chance this is related to stack overflows?

[vnen]

AFAIK it does not "crash" it just stops responding. If you click on the window then Windows will show the problem and asks if you want to force close it, which can be mistaken by a crash.

I still think it's just a matter of moving the scan to a new thread to show some progress and keep the editor responsive.

[akien-mga]

I still think it's just a matter of moving the scan to a new thread to show some progress and keep the editor responsive.

Edited issue title accordingly.

[Calinou]

Note that while #30735 helps with project loading speed, it doesn't implement non-blocking scanning.

If we want a simple solution to this, we can dim the window while projects are being scanned (like when quitting the editor or project manager). This will at least provide some kind of visual feedback without complexifying the code too much. We could also abort scanning if it takes more than 5 seconds or so.

[KoBeWi]

This doesn't need to run in thread actually. We could just utilize NOTIFICATION_PROCESS and e.g. scan 100 directories each frame. Easy to do and even allows cancelling, so it's more UX problem.

[Zylann]

If this is done in process, then it should be staggered (i.e keep scanning until 10 ms elapsed then continue next frame) because filesystem access can vary wildly across computers, hardware and OSes. On the other hand, with sliiightly more work, using a thread may actually be a more efficient approach while keeping UI perfectly responsive, and it can be cancelled even just with a boolean.
I did the staggered approach for Find in Files as well but I'd like to change it to a thread one day for larger projects.

[akien-mga]

Still valid as of 4.0 for the record. This is only a problem on huge folders, even a somewhat big tree of projects like https://github.com/godotengine/godot-demo-projects is scanned in milliseconds.

[kingthrillgore]

I see a similar behavior when I attempt to open a large project on a separate in-system HDD (its own disk letter) on Windows 10 with 4.1.2. I don't see a lot of logging data to help narrow this down, and after force closing I can reopen it and it will load the second time. It's a big project though, and if the developers would like to access the repo to aid in testing, I can make it available to them.