remove filtering of common strains during loading

[CunliangGeng]

The loading part should return all relevant strains to scoring part. And then users should do the filtering of strains in the scoring part. So removing the method _filter_only_common_strains from loading process.

The upcoming PRs will add the filtering of common strains in the scoring part.

[CunliangGeng]

Current dependencies on/for this PR:

• dev
  • PR refactor filtering of user specified strains #178
    • PR Enable GCF loader to filter singleton GCFs #181
      • PR return list for get_bgcs methods of BGC loaders #182
        • PR rename strains.py to strain.py #183
          • PR remove deprecated functions of loading genomics data #184
            • PR update the process of loading genomics data #185
              • PR rename genomics.py to utils.py #186
                • PR remove deprecated functions of loading metabolomics data #187
                  • PR refactor class MolecularFamily #188
                  • PR refactor class GCF #189
                  • PR enable parallel testing using pytest-xdist #190
                  • PR Enable filtering of singleton molecular families #191
                  • PR Remove class SingletonFamily and MF attribute id #192
                  • PR refactor Spectrum class #193
                  • PR remove unused mgf.py and its test file #194
                  • PR Update logics of loading metabolomics data #196
                  • PR Update loading of genomics data #197
                  • PR deprecate the method _load_class_info #198
                  • PR remove loading of optional data #199
                  • PR remove filtering of common strains during loading #200 👈
                  • PR update logics of loading mibig data #201
                  • PR refactor initiation of paths to metabolomics files #202
                  • PR Update logics of loading metabolomics data #195

This stack of pull requests is managed by Graphite.

[gcroci2]

Maybe I'm missing something here, but if a strain is not relevant (i.e., it's not present in both genomic and metabolomic data), what is the use case in which it could be used for the scoring part anyway? Which means, how can a link be present (and scored) at all in such cases?

[justinjjvanderhooft]

Good point @gcroci2, I think it depends on how we organize the loading of the strains - and the matching. This is one of the most critical steps as so far, most of the issues arise from this. @CunliangGeng, what is your current idea? First load everything and then ensure there are matches in a second step, before moving on to the scoring?

[CunliangGeng]

@gcroci2 @justinjjvanderhooft If a strain is not a common strain, it could be used by only genomics or only metabolomics or neither. The current workflow will load everything and pass them to scoring part, users then set what strains to use in the scoring part, e.g. there would be a parameter in scoring like use_only_common_strains and/or filter_specified_strains...

[justinjjvanderhooft]

@CunliangGeng I think I can follow your logic, as in, we load all that is there, and then prior to scoring we let the user choose/assess what will be considered....is that how you envisioned it? I think what @gcroci2 and myself are referring to is that, for any "linking score" to work, we need at least 1 (and preferably more!) "common" strains between the genome and metabolome data that is uploaded. For the PoDP data, this is not an issue, as they are matched through the PoDP metadata. For the local data (where many users will start with), what do we currently have in mind? Upon preparing the data, do a validation that at least one strain label is matching (common) between the genome and metabolome data? And/or give an overview of common and "not-matching" strains (labels) prior to calculating the score(s). It is good that you raise this point. I am happy to follow your advice/logic @CunliangGeng, as long as it is clear to the user in an early stage how many matching (common) strains there are available for any scoring. Whilst it is still unclear to me now what we can do with the not matching strain labels, I can think of enough use cases (i.e., streamlined loading of genome data for networking, etc.) - is that what you had in mind @CunliangGeng?

[CunliangGeng]

@justinjjvanderhooft @gcroci2
Here are my answers to Justin's questions:

If users set it to use common strains for scoring and actually there is no common strains, then it should raise an warning/error.

We can provide methods in class NPLinker to check if common strains exist or not, or to check the number of common/non-common strains.

For the local data (where many users will start with), what do we currently have in mind? Upon preparing the data, do a validation that at least one strain label is matching (common) between the genome and metabolome data?

For the local data mode, users have to manually prepare the strain mappings and ensure the strain id has alias for both genomics and metabolomics data. After loading the strain mappings (and also other data), users can use the new method in NPLinker class to check the number of different types of strains.

Whilst it is still unclear to me now what we can do with the not matching strain labels, I can think of enough use cases (i.e., streamlined loading of genome data for networking, etc.) - is that what you had in mind

For the data like BGC that do not have a matched strain, they won't be passed to scoring stage. In the loading stage, the number of matched/unmatched data will be reported in the log.

[justinjjvanderhooft]

Thanks for your answers @gcroci2! I think it may also be a "semantic" confusion, but I think I can follow the logic now. Let's get back to this in an online / in person meeting to clarify everything :-)

[CunliangGeng]

@gcroci2 Please explicitly approve the PR if it looks good. Without approval from at least one reviewer, the PR cannot be merged :-)

[CunliangGeng]

Merge activity

• Jan 24, 10:20 AM: @CunliangGeng started a stack merge that includes this pull request via Graphite.
• Jan 24, 10:28 AM: Graphite rebased this pull request as part of a merge.
• Jan 24, 10:29 AM: @CunliangGeng merged this pull request with Graphite.