ENH: automatically load external frontends in load()

[Xarthisius]

PR Summary

This PR modifies the behavior of yt.load to auto-include subclasses of Dataset from external, installed packages. External packages must define an entry point in yt.frontends linked to the Dataset subclass (see docs for an example).

PR Checklist

• New features are documented, with docstrings and narrative docs
• Adds a test for any bugs fixed. Adds tests for new features.

[neutrinoceros]

I had no idea this was possible, but I love it, thank you !

[neutrinoceros]

additional pointers. I haven't studied why the test is failing on Windows yet

[Xarthisius]

How to fix windows is beyond me... Why answer tests fail (with blank differences) is also beyond me. I think I addressed everything else.

[neutrinoceros]

How to fix windows is beyond me...

Am I right to assume fixing #4286 will not resolve the problem here ?
I will try to find a solution here.

Why answer tests fail (with blank differences) is also beyond me.

This is just flakiness, do not worry about it: it's been like this for months now, and we have a solution in #4122

[Xarthisius]

How to fix windows is beyond me...

Am I right to assume fixing #4286 will not resolve the problem here ? I will try to find a solution here.

I pushed the change, let's see.

[neutrinoceros]

Well played, looks like this now fixes #4286 , right ?
My two unresolved threads still stand, is it alright if I push one commit to add test cleanup, or would you prefer that I write the fixture as a comment ?

[Xarthisius]

My two unresolved threads still stand, is it alright if I push one commit to add test cleanup, or would you prefer that I write the fixture as a comment ?

You can push directly.

[neutrinoceros]

cool. I'll do this in the next couple hours

[neutrinoceros]

I will double check that it works as is for yt_idefix

[neutrinoceros]

It works perfectly with yt_idefix, thanks again for starting this !

[chrishavlin]

this is great!

I do think it will marginally increase the chance of having plugin loading conflicts. Since previously you had to explicitly import the plugin, you'd never end up with two external frontends identifying a file as valid. But now if a user happens to have installed two external entrypoint-enabled frontend plugins that both identify a file as valid they'll get an YTAmbiguousDataType error. But the benefit here far outweighs the slim chance of encountering that IMO so I don't think it's worth worrying about, but I thought it was worth mentioning.

[neutrinoceros]

But now if a user happens to have installed two external entrypoint-enabled frontend plugins that both identify a file as valid they'll get an YTAmbiguousDataType error. But the benefit here far outweighs the slim chance of encountering that IMO so I don't think it's worth worrying about, but I thought it was worth mentioning.

That's correct, although it's nice to think that this will not be a deadend situation even then thanks to yt.load's hint keyword argument !

[neutrinoceros]

pre-commit.ci autofix

[neutrinoceros]

marking this PR for backport but I'll only cherry pick b276292