Qt widget for loading pose datasets as napari Points layers

[niksirbi]

Description

What is this PR

• Bug fix
• Addition of a new feature
• Other

Why is this PR needed?
This is the 2nd in a series of multiple PRs (following #218), in which I'll try breaking down the work-in-progress contained in #112, and debug any obstacles I encounter along the way. These PRs will be merged into the napari-dev branch, until the plugin becomes minimally functional for users, at which point we'll merge the napari-dev branch into main.

This PR closes #47.

What does this PR do?
It replaces the placeholder "hello" Qt widget with an actual widget that can load movement poses dataset into napari as a Points layer, via functions defined in napari/convert.py. The position data variable is represented in the Points array, while confidence is stored in the layer properties, alongside individual and keypoint names. This means that when hovering over any point in napari, one can see these properties displayed.

In future PRs, the properties will be used to control the appearance of points (i.e. mapping various properties to point size, colour, etc.) For now, the points have a fixed style (coloured by individual ID for multi-individual datasets, by keypoint ID for single-individual datasets). To facilitate the consistent styling of these points, I've defined dataclasses in napari/layer_styles.py, where we can store the default styling per layer type.

Currently, the plugin with the poses loader widget looks like this:

What does this PR NOT do?
These features, though planned, are not essential enough to be part of the "napari prototype" #31:

• No loading of bboxes datasets napari widget for loading bboxes datasets from file #310
• No creation of a napari Tracks layer (that would enable us to display trajectory "tails") Visualise pose tracks as a napari Tracks layer #4

Code structure

The new modules all reside within the movement/napari folder, and include:

• _meta_widget.py: this creates a container of collapsible widgets (imported from brainglobe-utils). Within this container we can stack many widgets, each handling a different task/workflow step. All widgets except for the currently expanded (active) one will appear collapsed. For now this container only houses one widget - the poses loader (see next point).
• _loader_widgets.py: contains the PosesLoader class, which defines the first (and for now, only) collapsible widget. PosesLoader is essentially a frontend for the load_poses.from_file() function. When a file is being loaded, the _on_load_clicked method will call the poses_to_napari_tracks function (see next point) which does all the data wrangling required to transform a movement poses dataset into napari compatible array + properties.
• convert.py: which contains the aforementioned poses_to_napari_tracks function. The idea is that this module will house all movenet->napari and napari->movement conversion utilities. There's something I need to clarify here: why am I creating the data for a napari Tracks layer if my intention is to create a Points layer? That's because the data structure for both of these the same: Points are specified as an array with [z, y, x] columns (in out case that's [time, y, x]), whereas Tracks require columns [track_id, time, y, x]. So by creating a Tracks array, we maintain the track_id information (which we will need in the future anyway), and we get a Points array for "free" (by taking the last 3 columns).
• layer_styles.py: dataclasses defining styles for napari layers. Here I've created a base LayerStyle class with attributes that are common across layer types, and a PointsStyle child class with attributes specific to Points layers. The latter class implements a set_color_by method, which simplifies the re-coloring of the points according to a chosen property (e.g. keypoint or individual).

New unit test modules exactly mirror the above files (see below).

References

• Part of Make a napari frontend prototype #31.
• Preceded by Create skeleton for napari plugin with collapsible widgets #218
• To be followed by Write a guide on how to visualise poses in napari #283

After the current PR is merged, the only absolutely required issue for completing the prototype would be #283. After we have a guide to using the napari plugin, we can merge the prototype to main, and release v0.1 of movement.

How has this PR been tested?

Unit tests have been written for all new modules, and can be found in thetests/test_unit/test_napari_pluginfolder.
The test modules map 1-to-1 to the aforementioned new code modules, so:

• test_meta_widget.py
• test_poses_loader_widget.py
• test_convert.py
• test_layers_styles.py

There is one untested widget method, PosesLoader._on_browse_clicked which opens a file dialog to select a poses file. Despite spending lots of time on this, I couldn't figure out a way to successfully mock all the actions required for testing the file dialog, without actually opening one. Suggestions welcome. Alternatively, this can be opened as a separate issue. I basically decided to open this PR for review without this unit test, because the whole matter was dragging for far too long, and perhaps testing that widget is not so important.

How to review this PR

• Install movement from this branch with dev dependencies: pip install -e .[dev]. This will also include the optional dependencies specified under the napari extra.
• Launch napari with the movement plugin docked: napari -w movement.
• For manual testing of the plugin, it's easiest to use our sample data (locally found under ~/.movement/data).
• Use the "load poses" widget to load the corresponding poses file (for the EPM video, you may use any of the EPM files, from DLC or SLEAP).
• If you wish to visualise the poses against a background, you have two options:
  • Drag and drop a corresponding sample frame into napari (those can be found under ~/.movement/data/frames. This should always work, give you are using the right image for the poses file
  • Install the napari-video plugin, and drag and drop one of the videos found in ~/.movement/data/videos (choose to open it with the video plugin when napari prompts you). Initially I'd planned to include napari-video as a dependency, but I ran into issues during CI. We have to find a way around these as part of Napari plugin reader for videos #49. For what it's worth, the video plugin currently works fine in my M2 Macbook, is laggy on Ubuntu, and fails on Ubuntu CI. See below in "Known issues" for more info.

Known issues

• When displaying properties in the viewer (by hovering over a point), there are way too many decimal points shown for properties with float values. This has been filed as a napari issue.
• As mentioned above, the napari-video plugin currently doesn't perform reliably cross-platform, see related issues:
  • Napari plugin reader for videos #49
  • Why not use opencv-python-headless also for newer versions of Python? postpop/videoreader#6
  • inaccurate seeking janclemenslab/napari-video#3
• Codecov complains because of the aforementioned lack of tests for PosesLoader._on_browse_clicked

Because reviewing this PR will take a long time, I suggest @lochhh and @sfmig you start to slowly work your way through it, while I focus on solving these known issue in parallel.

Is this a breaking change?

No.

Does this PR require an update to the documentation?

Yes, but this will be done as part of #283.

Checklist:

• The code has been tested locally
• Tests have been added to cover all new functionality (almost!)
• The documentation has been updated to reflect any changes
• The code has been formatted with pre-commit

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov]

Codecov Report

Attention: Patch coverage is 95.07042% with 7 lines in your changes missing coverage. Please review.

Project coverage is 99.12%. Comparing base (fae3f25) to head (254c5f9).

Files with missing lines	Patch %	Lines
movement/napari/_loader_widgets.py	91.25%	7 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff               @@
##           napari-dev     #253      +/-   ##
==============================================
- Coverage       99.77%   99.12%   -0.66%     
==============================================
  Files              16       18       +2     
  Lines             908     1034     +126     
==============================================
+ Hits              906     1025     +119     
- Misses              2        9       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[niksirbi]

Regarding the un-tested PosesLoader._on_browse_clicked widget, @IgorTatarnikov has pointed me to some tests he wrote for a similar widget here. I should be able to adapt them for our use-case.