Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.19.0 #203

Merged
merged 128 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
1297a72
Fix some typos in the human-readable assay region names
fbdtemme Aug 20, 2024
5dcad08
Only install dev packages in dev container layer
fbdtemme Aug 20, 2024
622cb6f
Merge pull request #180 from PixelgenTechnologies/misc-small-fixes
fbdtemme Aug 20, 2024
dd98043
Added abundance_colocalization_plot function
ptajvar Aug 20, 2024
0aefc35
added test for abundance_colocalization_plot
ptajvar Aug 21, 2024
a4ab4dd
added plot legend explanation for abundance_colocalization_plot
ptajvar Aug 22, 2024
334d78c
updated CHANGELOG.md
ptajvar Aug 22, 2024
4b96a62
Fix tox and modernize tests.yml worfklow
fbdtemme Aug 22, 2024
9e1afbd
Merge pull request #179 from PixelgenTechnologies/feature/exe-1852-sc…
ptajvar Aug 22, 2024
03baf4a
Remove leftover workflow step
fbdtemme Aug 22, 2024
1299abe
Silence a typing warning
fbdtemme Aug 22, 2024
9d624d0
Merge pull request #184 from PixelgenTechnologies/fix-tox
fbdtemme Aug 23, 2024
1963fa8
Rename `single-cell` command to `single-cell-mpx` and make it an alias
fbdtemme Aug 20, 2024
911c222
Update workdir and cli info to deal with both `single-cell-mpx` and `…
fbdtemme Aug 22, 2024
1d2010e
Update tests
fbdtemme Aug 22, 2024
0c17c13
Update CHANGELOG.md
fbdtemme Aug 22, 2024
235d84f
Fix typos `snipped` -> `snippet`
fbdtemme Aug 22, 2024
1fe31df
Remove unused import
fbdtemme Aug 22, 2024
effa856
Fix a typo in `CLIInvocationInfo.get_option`
fbdtemme Aug 22, 2024
59744e1
Revert single-cell-mpx entrypoint renaming
fbdtemme Aug 22, 2024
e2b4679
Revert CHANGELOG change
fbdtemme Aug 22, 2024
88657ac
Revert testdata update
fbdtemme Aug 23, 2024
6b21513
Merge pull request #182 from PixelgenTechnologies/feature/exe-1954
fbdtemme Aug 23, 2024
36b6002
add support for multiple targets in get_differential_colocalization
ptajvar Aug 22, 2024
51c025e
added support for multiple targets in plot_colocalization_diff_heatmap
ptajvar Aug 22, 2024
7dfd750
fixed output type for plot_colocalization_diff_heatmap
ptajvar Aug 22, 2024
f14729b
updated colocalization heatmap and volcano plots
ptajvar Aug 26, 2024
28b6daf
fix: replaced p_adj and target columns in test_get_differential_coloc…
ptajvar Aug 26, 2024
94a908e
added plot titles to differential colocalization volcano plots
ptajvar Aug 26, 2024
3d1fd36
updated CHANGELOG.md
ptajvar Aug 26, 2024
d9fa121
use_z_score is replaced by value_column in colocalization plot functi…
ptajvar Aug 26, 2024
86372e2
added plot_polarity_diff_volcano
ptajvar Aug 27, 2024
68074e6
Merge pull request #183 from PixelgenTechnologies/feature/exe-1935-su…
ptajvar Aug 27, 2024
50261bb
Merge pull request #185 from PixelgenTechnologies/feature/exe-1936-ad…
ptajvar Aug 27, 2024
4b7b687
Updated probability distance calculations
ludvigla Aug 30, 2024
ddb36a9
Added test for multigraph
ludvigla Aug 30, 2024
e097b1f
updated CHANGELOG
ludvigla Aug 30, 2024
8bbabb5
Fixed typo
ludvigla Aug 30, 2024
c9baf9b
skip casting nodes and edges to list
ptajvar Sep 2, 2024
8d05475
Merge pull request #186 from PixelgenTechnologies/exe-1997-improve-wpmds
ptajvar Sep 2, 2024
805f2e7
Relax pyarrow dependency to >=14
fbdtemme Sep 10, 2024
8526c31
Merge pull request #187 from PixelgenTechnologies/update-pyarrow
fbdtemme Sep 10, 2024
ed0d201
fix: multi-target polarity plot
ptajvar Sep 16, 2024
848fa42
Merge pull request #189 from PixelgenTechnologies/feature/exe-2009-fi…
ptajvar Sep 16, 2024
6e46177
midway
ptajvar Sep 9, 2024
7c8fa9f
midway2
ptajvar Sep 9, 2024
8a2fbe3
midway
ptajvar Sep 13, 2024
1629fbc
added xxhash to dependencies
ptajvar Sep 13, 2024
b814b6e
updated poetry.lock
ptajvar Sep 13, 2024
60765db
revert poetry.lock
ptajvar Sep 13, 2024
7534a09
revert poetry.lock
ptajvar Sep 13, 2024
3c8c4ca
fix: typo
ptajvar Sep 13, 2024
5c1deb2
updated poetry.lock
ptajvar Sep 13, 2024
6e68c29
adapting tests
ptajvar Sep 16, 2024
075b4d8
make regular expression match compatible with python 3.10
ptajvar Sep 16, 2024
828da76
fix: split_remaining_and_removed_edgelist output type in graph utils
ptajvar Sep 16, 2024
19dd958
add LEIDEN_RESOLUTION as a constant parameter
ptajvar Sep 17, 2024
bd36169
improve map_upis_to_components speed
ptajvar Sep 17, 2024
744aa90
rerun plot tests
ptajvar Sep 17, 2024
50747f0
applied suggestions from pull request review
ptajvar Sep 17, 2024
6496116
using pandas edges instead of polars for improved speed in recover_te…
ptajvar Sep 20, 2024
bcde2c0
doing 5 trials for each leiden run
ptajvar Sep 24, 2024
2eec02b
lower leiden resolution to 0.01 and increase the community rejoin thr…
ptajvar Sep 24, 2024
43250d5
fix: added back graph report generation
ptajvar Sep 24, 2024
fa18bfb
Update src/pixelator/graph/community_detection.py
ptajvar Sep 25, 2024
5a3a175
Update src/pixelator/graph/community_detection.py
ptajvar Sep 25, 2024
c1c2b68
Update src/pixelator/graph/community_detection.py
ptajvar Sep 25, 2024
4c372af
Applied the review comments
ptajvar Sep 26, 2024
ed74a2d
removing components_modularity from edgelist_metrics
ptajvar Sep 30, 2024
e3e2c06
Merge pull request #192 from PixelgenTechnologies/feature/exe-2015-ed…
ptajvar Oct 1, 2024
14086e4
fix: format
ptajvar Oct 1, 2024
6884aa9
reducing STRONG_EDGE_THRESHOLD to 5
ptajvar Oct 1, 2024
a2e3bf9
stub clarification
ptajvar Oct 1, 2024
a477cba
Update src/pixelator/graph/community_detection.py
ptajvar Oct 1, 2024
d0ff1d9
adding number of edges with colliding UPIs to graph report.json
ptajvar Oct 1, 2024
62b7cbc
update CHANGELOG.md
ptajvar Oct 2, 2024
5b4df56
replaced the constant STRONG_EDGE_THRESHOLD with --max-edges-to-split…
ptajvar Oct 2, 2024
2a43b72
Merge pull request #188 from PixelgenTechnologies/feature/exe-1989-it…
ptajvar Oct 2, 2024
cad67e3
Update fspec, and update lock file
johandahlberg Oct 18, 2024
9246b33
Merge pull request #194 from PixelgenTechnologies/chore/update-fsspec
johandahlberg Oct 18, 2024
a2d0fed
fix: changelog merge conflict
ambarrio Oct 20, 2024
8ad1a9b
adding annotation of potential doublets
ptajvar Oct 15, 2024
e08516d
split _compute_sub_communities from _assess_doublet
ptajvar Oct 21, 2024
128f116
Added n_edges_to_split_potential_doublets
ptajvar Oct 21, 2024
42b1537
Reduced Leiden resolution by 0.5 in the annotation step
ptajvar Oct 21, 2024
7a3e5c1
updated CHANGELOG.md
ptajvar Oct 21, 2024
2b03001
fix: set default values for fraction_potential_doublets and n_edges_t…
ptajvar Oct 21, 2024
4d38f39
fix: make fraction_potential_doublets non-optional
ptajvar Oct 21, 2024
13f6618
Update src/pixelator/pixeldataset/utils.py
ptajvar Oct 22, 2024
a427c36
Update src/pixelator/pixeldataset/utils.py
ptajvar Oct 22, 2024
19bd66a
Update src/pixelator/pixeldataset/utils.py
ptajvar Oct 22, 2024
2e46451
Update src/pixelator/pixeldataset/utils.py
ptajvar Oct 22, 2024
8db21e2
fix: typo
ptajvar Oct 22, 2024
98da20f
added RELATIVE_ANNOTATE_RESOLUTION as a constant
ptajvar Oct 22, 2024
69041e7
make fraction_potential_doublets and n_edges_to_split_potential_doubl…
ptajvar Oct 22, 2024
63b4ef1
set default fraction_potential_doublets and n_edges_to_split_potentia…
ptajvar Oct 22, 2024
fb156c9
explicitly check n_edges_to_split_doublet exists before using sum
ptajvar Oct 22, 2024
7042c1f
fix: typecheck
ptajvar Oct 22, 2024
7ca7168
Merge pull request #193 from PixelgenTechnologies/feature/exe-2025-an…
ptajvar Oct 22, 2024
436867f
Add "depth" column to the discarded_edgelist
ptajvar Oct 22, 2024
3d4cc74
updated CHANGELOG.md
ptajvar Oct 22, 2024
b63fb91
Added multiplet recovery metrics to graph stage's report.json
ptajvar Oct 22, 2024
dde6fbe
fix: typecheck
ptajvar Oct 22, 2024
37d1ddf
fix: Typechek
ptajvar Oct 22, 2024
63145f7
casting report parameters to corresponding data types
ptajvar Oct 22, 2024
ffa1b8d
Merge pull request #195 from PixelgenTechnologies/feature/exe-2046-ad…
ptajvar Oct 22, 2024
3027643
Do dynamic imports of graspologic for performance
johandahlberg Oct 28, 2024
d9e4022
Merge pull request #196 from PixelgenTechnologies/chore/dynamic-impor…
johandahlberg Oct 28, 2024
dbfbd98
casting component to string before checking for match when constructi…
ptajvar Nov 5, 2024
9834f48
Merge pull request #197 from PixelgenTechnologies/fix/exe-2058-graph-…
ptajvar Nov 6, 2024
56322c8
Enforce more nodes than dimensions for pmds
johandahlberg Nov 13, 2024
112b41c
Add test
johandahlberg Nov 13, 2024
b25a7ec
Update changelog
johandahlberg Nov 13, 2024
5d1bd31
Exit with status 1 if less 50% of reads demuxed
johandahlberg Nov 13, 2024
d3172c6
Update changelog
johandahlberg Nov 13, 2024
433e234
Merge pull request #198 from PixelgenTechnologies/feature/exe-2061-re…
johandahlberg Nov 13, 2024
d07eace
Merge pull request #199 from PixelgenTechnologies/feature/exe-2054-ba…
johandahlberg Nov 14, 2024
0a38a6f
Raise error when dynamic and hard filters both on
johandahlberg Dec 10, 2024
26cc9e7
Update changelog
johandahlberg Dec 10, 2024
8a5df55
Fix typo
johandahlberg Dec 10, 2024
542cc28
Merge pull request #200 from PixelgenTechnologies/feature/exe-2075-fa…
johandahlberg Dec 10, 2024
3845df2
Set minumum threshold for the dynamic size filter
johandahlberg Dec 10, 2024
9899d56
Better error message when all components filtered
johandahlberg Dec 10, 2024
45a8cdd
Merge pull request #201 from PixelgenTechnologies/feature/exe-2065-se…
johandahlberg Dec 10, 2024
577f567
fix: changelog
ambarrio Dec 10, 2024
6fe1062
fix: changelog
ambarrio Dec 10, 2024
702c122
Update CHANGELOG.md
ambarrio Dec 10, 2024
243cc4c
Merge pull request #202 from PixelgenTechnologies/release-0.19
ambarrio Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 23 additions & 32 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: Set up Python environment - ${{ matrix.py }}
- name: Set up Python environment
uses: ./.github/actions/setup-python
with:
python-version: ${{ matrix.py }}
Expand All @@ -69,30 +69,19 @@ jobs:
with:
uri: 'https://opengene.org/fastp/fastp'
name: 'fastp'
version: '0.23.2'
version: '0.23.4'

- name: Generate tox env for python version
uses: actions/github-script@v6
id: set_tox_env
with:
script: |
const pythonVersion = "${{ matrix.py }}"
const toxEnv = "py" + pythonVersion.replace(/\./g, '')
core.setOutput("tox_env", toxEnv)
- name: Install and configure Poetry
uses: snok/install-poetry@v1

- name: Install dependencies
run: |
python -m pip install --user pipx
python -m pipx ensurepath
pipx install poetry
pip install tox
poetry self add poetry-plugin-export
poetry export --output dev_requirements.txt --quiet --no-interaction --no-ansi --without-hashes --with dev
pip install -r dev_requirements.txt
pip install -e .

- name: Run test suite on local python version
run: tox -e ${{ steps.set_tox_env.outputs.tox_env }} --current-env
- name: Install tox
run: python -m pip install tox-gh>=1.2

- name: Setup test suite
run: tox -vv --notest

- name: Run test suite
run: tox --skip-pkg-install

test-coverage:
name: "Check the test coverage"
Expand All @@ -109,15 +98,16 @@ jobs:
with:
python-version: "3.11"

- name: Create dependencies file
- name: Install and configure Poetry
uses: snok/install-poetry@v1

- name: Install dependencies
run: |
pip install poetry
poetry export --output requirements.txt --without-hashes --no-interaction --no-ansi --with dev
pip install -r requirements.txt
poetry install --with dev

- name: Run test suite
run: |
pytest --cov=src/
poetry run pytest --cov=src/

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
Expand Down Expand Up @@ -150,12 +140,13 @@ jobs:
name: 'fastp'
version: '0.23.2'

- name: Install and configure Poetry
uses: snok/install-poetry@v1

- name: Install dependencies
run: |
pip install poetry
poetry export --output requirements.txt --without-hashes --no-interaction --no-ansi --with dev
pip install -r requirements.txt
poetry install --with dev

- name: Run workflow tests
run: |
pytest -s -m "workflow_test" --basetemp=/tmp/pytest-results tests/integration/test_small_${{ matrix.case }}.yaml --log-cli-level=DEBUG
poetry run pytest -s -m "workflow_test" --basetemp=/tmp/pytest-results tests/integration/test_small_${{ matrix.case }}.yaml --log-cli-level=DEBUG
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,41 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.19.0] - 2024-12-10

### Changed

- Name components using UPI hashes.
- Run multiple iterations of multiplet recovery during the `graph` stage specified using `--max-refinement-recursion-depth`.
- Specify maximum number of edges that can be removed between two sub-components during multiplet recovery using `--max-edges-to-split`.
- Support for MultiGraphs in `pmds_layout`
- Support multiple targets in `plot_colocalization_diff_volcano` and `plot_colocalization_diff_heatmap`.
- If demultiplexing has a success rate lower than 50% the command will exit with a status of 1. This prevents further pipeline stages to be run on
what is probably bad data.
- Clarify that `--min-size` and `--max-size` in the `annotate` stage should not be used at the same time as `--dynamic-filter`.
- Setting a lower threshold of 300 edges when `--dynamic-filter` is used in the `annotate` stage, components smaller than that will always
be filtered. Note that this can still be overridden by setting `--min-size` explicitly.
- Clarify error message when all components are filtered out in the `annotate` stage.

### Added

- Add `depth` column to `discarded_edgelist.parquet` output of the `graph` stage that indicates at which refinement iteration the edge is removed.
- Add `edges_removed_in_multiplet_recovery_first_iteration`, `edges_removed_in_multiplet_recovery_refinement` and `fraction_edges_removed_in_refinement` to graph report.json.
- Add `is_potential_doublet` and `n_edges_to_split_doublet` columns to adata.obs.
- Add `fraction_potential_doublets` and `n_edges_to_split_potential_doublets` to annotate report.json.
- Add `--max-edges-to-split` option to `graph` to specify the maximum number of edges that can be removed between two sub-components during multiplet recovery.
- Add `abundance_colocalization_plot` function to make scatter plots of selected marker-pairs' abundance.
- Add `plot_polarity_diff_volcano` to make statistical comparison plots of selected component groups.
- Add `get_differential_polarity` to statistically compare polarity scores of selected component groups.

### Removed

- Remove the `components_recovered.csv` output from the `graph` stage.

### Fixed

- Improved error message when the number of nodes is lower than the number of requested dimensions in `pmds_layout`.

## [0.18.3] - 2024-09-26

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion containers/dev.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ WORKDIR /pixelator
COPY ./ /pixelator
COPY .git /pixelator/.git

RUN poetry export --output requirements.txt --without-hashes --no-interaction --no-ansi --with dev
RUN poetry export --output requirements.txt --without-hashes --no-interaction --no-ansi --only dev

RUN pip3.11 install -r requirements.txt && rm requirements.txt
Loading
Loading