RFC: decouple yt.config from initialization

[neutrinoceros]

PR Summary

This patch postpones the "live" part of the configuration to the very end of yt.__init__ and disentangle the configuration code a bit. This is achieved by

• never importing yt.config at the top of any module. Import statements are relocated to inside functions that need to access the global yt.config.ytcfg object. This is in line with the existing practice in the yt.funcs module
• making yt.config depend on yt.utilities.configure (instead of the other way around). This means a sizeable part of the diff here is simply due to relocating the YTConfig class
• adding a small callback system in yt.utilities.configure, allowing any module to perform configuration at runtime, but delaying it until yt.config is imported from for the first time (in yt.__init__)

The first commit fixes #3625 while the second fixes #3045

It also has obvious conflicts with #3106, but I think it'd be easier to merge the present PR first.

[neutrinoceros]

After I managed postponing configuration to the end of yt.init, I was able to fix #3045 trivially

[neutrinoceros]

pre-commit.ci autofix

[matthewturk]

This seems like a good refactor; your third bullet point is intriguing to me though and I'd like to hear if you have anything in mind. If you fix the conflicts I think it's good to go.

[neutrinoceros]

your third bullet point is intriguing to me though and I'd like to hear if you have anything in mind.

I'm not promoting this callback system as a feature-enabling change, it's mostly there to allow fixing the issues I was looking into without breaking backwards compatibility.

If you fix the conflicts I think it's good to go.

solved !

[neutrinoceros]

Forced pushed. Marking as draft until CI passes.

[matthewturk]

@neutrinoceros Do you think it is worthwhile to explore a wrapper function (for our wrapper function!) that manages the imports appropriately? Something like how we do on-demand imports, or something like that. I am slightly anxious about the increased verbosity of adding from yt.config import ytcfg inside all the functions; it's not bad but I think it might discourage using the config system at all.

[neutrinoceros]

I don't have a clear idea how to do that, and I'm convinced that I can come up with a different solution that will not add technical debt :/

[neutrinoceros]

Actually here's an idea to avoid having to nest imports:
the callbacks may be run explicitly by calling a master function living inside yt.config, so it'd be easy to control when that function is called (at the end of yt.__init__). I've tested this locally and can easily update this PR. Would that work for you ?

[matthewturk]

@neutrinoceros I think that would be great!

[neutrinoceros]

Ok for today I'll just resolve conflicts and switch this back to draft. I'll get back to this.

[neutrinoceros]

Switching to draft to give priority to #3831, since these are conflicting but only #3831 needs backporting

[matthewturk]

@neutrinoceros Should we re-up this?

[neutrinoceros]

@matthewturk conflict solved

[neutrinoceros]

@matthewturk did you mean to approve this ?