RFC: a more robust plot norm/colorbar API

[neutrinoceros]

PR Summary

This is a deep refactor of how colorbar and plot norm data are stored and utilized internally.
It allows for a much more robust API where chaining operations like set_zlim and set_unit doesn't ruin the whole viz (fix #2538).
It also makes room to implement stuff like custom norm (fix #3840), and helps resolve a documented quirky behaviour (fix #3852).

It unifies the various strategies scattered across plot classes to determine what norm can be used in a given situation (symlog, log, linear ?), which is a area where we've seen many bugs in the recent pastn, basically on every matplotlib feature release.

I've been iterating and testing on this for a couple weeks locally, I'm opening the PR while it's still hot to see what bug existing tests might reveal.
Documentation is already included, but for now I haven't committed my test files because they are not adapated to yt's image comparison testing framework yet.

Note that this includes the changes from #2504, which changes some phase plots with respect to their gold standard, so CI should fail at the very least there, but no other failures are desired.

edit: I'm bumping matplotlib's minimal required version to 3.1 because 2.2.3 has some incompatibilities with the refactor and 3.1 is the oldest testable version for macOS.
FTR Matplotlib 3.1 was released in May 2019 and was only distributed for Python 3.6 and 3.7, so it's still a pretty conservative requirement now.

Note to reviewers

I opened a "companion" PR at #3900, which contains only the new tests, not the patch, to help demonstrate what's working here and not on the main branch.

I also tried to keep the commit history somewhat tidy and readable, so reading commits by chronological order is one possible angle to review this in steps.

After a lot of iterations, I bumped/added new answers both on Jenkins and the answer store. The supporting PR is yt-project/answer-store#31

A Summary of user-facing changes

A lot of symlog-related code is now centralised, making it reusable to all PlotContainer subclasses (not just ImagePlotContainer).

api changes

• updated ImagePlotContainer.set_zlim to :
  • allow passing zmin or zmax only (previously they had to be both specified somehow)
  • allow passing unit-aware values (either as unyt_quantity objects or parsable tuples e.g. (10, "g/cm**2"))
• updated PlotContainer.set_log so that the second argument (log) isn't necessary when passing linthresh. Previously users had to provide a value to the log argument but it was not actually used. Now a warning is emitted if a log value is received but discarded (this is however not deprecated).
• added a PlotContainer.set_norm method to allow using matplotlib norms other than lin, log and symlog

deprecations

• passing zmin=None (resp zmax=None) to set_zlim explicitly is deprecated (users can leave the parameter unset or pass zmin="min" (resp zmax="max") explicitly instead)
• PlotContainer.set_log's symlog_auto parameter is deprecated (the same behaviour can be obtained by passing linthresh="auto" instead)

dependencies

• bumped minimal supported version of MPL from 2.2.3 to 3.1 (this represents less than a 1yr jump)
• added a dependency to typing_extensions (which are backports from the standard lib's typing module), exclusively for Python 3.7 (we can rely fully on the std lib in 3.8 and beyond)

PR Checklist

• New features are documented, with docstrings and narrative docs
• cleanup remaining todos in the code (TODO(clm))
• Adds a test for any bug fixed. Adds tests for new features.
• cleanup history to facilitate reviews
• check that BUG: cannot plot data containing both finite and infinite values and use set_zlim #3901 is (or stays) fixed
• add a test for it

The following PRs should be handled before this one:

• Add masking for phase plots. #2504
• BUG: don't colorize pixels that are not completely within the data domain in pixelize_cylinder routine #3818
• BUG: fix a bug where setting zmin/zmax on a 2D plot with symlog norm would cause an undesired change in linthresh #3888

[matthewturk]

A few thoughts and questions before running off!

[matthewturk]

Is symlog_auto still allowed?

[neutrinoceros]

yes, but I'm deprecating it

[matthewturk]

Are these functions inspired by or taken from somewhere else?

[neutrinoceros]

It's all moved code:

• the get_x_minorticks routines are taken (moved) from yt.visualization.plot_container
• get_symlog_majorticks is extracted from ImagePlotMPL._init_image
• get_default_from_config is adapted from a closure in PlotContainer.setup_defaults

The may reason for moving them around was to help me remembering were everything I needed was defined, but I may restore some of them to their original location when I try to clean this branch a bit.

[neutrinoceros]

Some desired changes in image tests collide with #3818, which is set for backporting, so I'll wait for that PR to resolve before I open this one for review.

[neutrinoceros]

I think I got this in a state were all remaining failures are expected changes. I simplified the branch history and reduced the amount of commits by a factor 10.
I also listed 2 PRs that should be handled before this one in the OP.

[neutrinoceros]

Just noticed that some docs example like https://yt-project.org/doc/cookbook/simple_plots.html#showing-and-hiding-axis-labels-and-colorbars
are still broken on this branch. I'll continue to work on it.

[neutrinoceros]

Docs build got stuck
@yt-fido test this please

[chummels]

Again, many apologies for this taking so long to review. In general, I like what is done here. The old methods for specifying colorbar normalizations were getting pretty clunky after I added the auto_symlog kwarg a while back, and this is definitely an improvement over that. I tried to review as much as I could, but there is a lot of code here. In general, I just ran some test scripts locally to ensure that they worked and produced what was expected and read the code looking for any obvious inconsistencies.

My main comments are on how this is documented to ensure the users know how to use it, since it's changing the defaults a bit. I was hoping we could change references to "norms" in the docs to "normalizations" or "scalings", since I think norms is a bit confusing. But I realize that the classes and methods you created reference "Norms", which I think is fine. What do you think? After that minor stuff gets addressed, I'm happy to approve and merge this.

[chummels]

Have profile plots also moved to the new format--from below it looks like it? Maybe mentioning that here too?

[neutrinoceros]

Profile plots don't have a colorbar, unless you mean specifically 2D profile plots, aka phase plots ? (and yup, should mention it here)

[neutrinoceros]

@chummels thanks ! I think I replied to all your comments so far, but I'll let you resolve the threads when you're satisfied.

Just a note that merging this will not be a one step process. We need to merge the answer-store PR first, then update here.

[chummels]

Thanks for addressing my concerns. Psyched to see this go in! Only a couple of little requests, but other than that, this is great.

[neutrinoceros]

Alright, looks like we're go ! Thank you so much for going through this gigantic PR
I'm going to merge yt-project/answer-store#31 and update this to point to a long lived branch of the answer store

[neutrinoceros]

Ok this is now ready at last ! @chummels, if you'll do the honours ? 😄

[neutrinoceros]

The docs build failed, but I'm hoping it's a random server-side event:

RuntimeError: Kernel didn't respond in 60 seconds

Let's try again: @yt-fido test this please