Add SingleR and CellAssign metadata

[allyhawkins]

Closes #279 and starts to address #384

As I started to work on #384, the first thing that I had to figure out was how we are going to store the metadata for the references used with CellAssign, and how will that relate to the way we currently store the references for SingleR. Previously, we had been keeping a list of references to use with SingleR in ScPCA-admin, but there is no reason to me why the list of available references for use with the workflow should be there and not in this repo. This is similar to the metadata we now have for which references are available for single-cell (ref-metadata.tsv). That data is stored in the references folder as part of the repo, so here I am porting over the celltype-reference-metadata.tsv and making follow-up changes to account for that.

• In preparation for having references for both SingleR and CellAssign, I added in new columns to account for the celltype_method. Now we are storing both what package/ database the references come from and the cell type method. I use that column to split the channel in the workflow.
• Related to the above point, I added an organs column that will be used to indicate which organs from PanglaoDB should be grouped for each ref_name/ tissue type. I included muscle, brain, and blood right now as those are the most straight forward to me and probably going to be the most highly used refs anyways.
• I added a process to grab the references from celldex and save them (as was the original purpose of Create independent workflow for grabbing and saving refs for SingleR #279). The output is the rds file containing the celldex reference and becomes the input to the training process we already had.
• Again to prep for CellAssign and addressing Process for creating marker gene refs to use with CellAssign #384, I created the channel that will become input to a process for generating marker gene reference matrices. I plan on creating the process and the script that gets run to build the marker gene matrices as a separate PR.
• I also added the PanglaoDB tsv file to the references folder.
• I haven't been able to test this fully, because I need celldex to be part of scpcaTools. Once that is in, I can test this and request a formal review.

[allyhawkins]

Just noting that this has now been tested and works successfully. The saved celldex refs can be found in the s3://scpca-references/celltype/singler_references folder and then models are in singler_models.

[sjspielman]

Looks good overall! I mostly have organizational and naming comments. Note that I did not test this myself at this stage, but plan to do so on 2nd round of review (also in part to test that all the suggestions, should you decide to accept them, are ok!).

• Since references now contains references of two different flavors for for two different workflows (e.g. transcriptome references vs cell type annotation references), time to add a quick references/README.md. (Later, we'll want more docs about, for example, adding more references to this workflow; can we open an issue for that?) For this README.md, here's some particular aspects I think we should make sure are there, beyond the baseline "what are these files" content -
  • Indicate that the organ column in celltype-reference-metadata.tsv has relevant values from the organ column in the PanglaoDB
  • References (no pun intended, probably, ok maybe a little) for celldex and PanglaoDB
• I'm not sure the variable name (and its "downstream" variables celltype_ref_database) is the right name here, since celldex isn't exactly a database (though PanglaoDB is). I wondered if "source" might be a better overall term here? If you agree, I've made suggestions throughout to use the term "source" instead of "database", e.g. celltype_ref_source in the TSV and ref_source in the workflow. Or, if you strongly disagree, I'm not offended if you resolve/ignore all these suggestions!
• I wonder if we want real params for singler_models/ and singler_references/ paths, e.g.

  # path in save_singler_refs() process
  params.singler_references_dir = "${params.celltype_ref_dir}/singler_references" 
  # path in train_singler_models() process
  params.singler_models_dir = "${params.celltype_ref_dir}/singler_models"
  

  This will make our lives easier should these values ever have to change, since it would all be defined in the references config and not directly within the workflow

[allyhawkins]

Since references now contains references of two different flavors for for two different workflows (e.g. transcriptome references vs cell type annotation references), time to add a quick references/README.md. (Later, we'll want more docs about, for example, adding more references to this workflow; can we open an issue for that?) For this README.md, here's some particular aspects I think we should make sure are there, beyond the baseline "what are these files" content -

I definitely agree we need docs, but I think that it makes more sense to do it in a separate issue, so I filed #389.

I'm not sure the variable name (and its "downstream" variables celltype_ref_database) is the right name here, since celldex isn't exactly a database (though PanglaoDB is). I wondered if "source" might be a better overall term here? If you agree, I've made suggestions throughout to use the term "source" instead of "database", e.g. celltype_ref_source in the TSV and ref_source in the workflow. Or, if you strongly disagree, I'm not offended if you resolve/ignore all these suggestions!

I liked this idea! So I went ahead and took your suggestions. I knew database wasn't right either, but couldn't find the right word, so thank you.

I wonder if we want real params for singler_models/ and singler_references/ paths, e.g

I had thought about this and was on the fence a lot. But I think you're right that changing them later will be easier to manage by making them params. Additionally, I would like to ultimately make a json file to hold the refs/paths that we have for cell typing just like we do for the regular workflow.

@sjspielman This is ready for another look.

[sjspielman]

LGTM! I just ran build-celltype-ref.nf, and all looks as expected in s3://scpca-references/celltype. My only comment is...

Additionally, I would like to ultimately make a json file to hold the refs/paths that we have for cell typing just like we do for the regular workflow.

Makes sense to me! Should this be an issue too (and we do should it come before or after #389? - your call!)?