Implementing DeepProfiler single cell output and normalization

[roshankern]

Description

As of commit e1091b3, DeepProfiler saves features in the format features/plate/well/site.npz This change allows pycytominer.cyto_utils.DeepProfilerData to work with this format with filename_delimiter = "/". A DeepProfilerData object can be used with AggregateDeepProfiler to aggregate the data (as in current version of pycytominer) or with SingleCellDeepProfiler to load and normalize single cell embeddings.

What is the nature of your change?

• Bug fix (fixes an issue).
• Enhancement (adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).
• This change requires a documentation update.

Checklist

Please ensure that all boxes are checked before indicating that a pull request is ready for review.

• I have read the CONTRIBUTING.md guidelines.
• My code follows the style guidelines of this project.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• My changes generate no new warnings.
• New and existing unit tests pass locally with my changes.
• I have added tests that prove my fix is effective or that my feature works.
• I have deleted all non-relevant text in this pull request template.

@gwaybio This is ready for review.

[gwaybio]

Looks great - I'm excited to have this functionality in pycytominer!

I have only one feature request. Can you add the functionality to output the non-normalized single cell features inside setup_normalize()? (I've suggested a rename to get_singlecells())

I've made several suggestions to making this change. I haven't tested these suggestions so please make sure that I've thought all the logic through.

I've also just now kicked off the continuous integration tests. If these pass then the only change required for the test suite is to add a test for the new output single cell functionality.

[roshankern]

Awesome @gwaybio! The changes in 836bf1e should resolve the integration test failures.

[gwaybio]

Congrats on the successful integration tests!

Two other comments, see below. Then I think we can merge.

[gwaybio]

Can you add a test where you apply normalize() to single_cells as described in 43 using the exact same normalize logic used in .normalize_deep_single_cells() and pd.DataFrame.assert_equal?

It would be good to test the internal normalize logic has expected behavior as an external application.

[roshankern]

Unfortunately I had to do a bit of strangling with the locations to make the DP data compatible with normalize(). Let me know if the changes in b754251 work for the pandas assertion.

[codecov-commenter]

Codecov Report

Merging #210 (34b4aba) into master (5f9a774) will decrease coverage by 0.23%.
The diff coverage is 91.96%.

@@            Coverage Diff             @@
##           master     #210      +/-   ##
==========================================
- Coverage   95.74%   95.51%   -0.24%     
==========================================
  Files          53       53              
  Lines        2611     2695      +84     
==========================================
+ Hits         2500     2574      +74     
- Misses        111      121      +10     

Flag	Coverage Δ
unittests	95.51% <91.96%> (-0.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pycytominer/cyto_utils/load.py	83.60% <66.66%> (-4.40%)	⬇️
pycytominer/cyto_utils/DeepProfiler_processing.py	92.96% <91.22%> (-3.63%)	⬇️
pycytominer/cyto_utils/__init__.py	100.00% <100.00%> (ø)
...ts/test_cyto_utils/test_DeepProfiler_processing.py	100.00% <100.00%> (ø)
pycytominer/tests/test_cyto_utils/test_load.py	100.00% <100.00%> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more