Discussion: Should we change the merged publish directory to be the given project directory?

[sjspielman]

This came up when discussing the scpca test data with @arkid15r. The question was, why aren't we sending merged projects into the project directories, and instead sending into a separate merged directory?

scpca-nf/merge.nf

Line 33 in 2e65200

publishDir "${params.results_dir}/merged/${merge_group_id}"

My sense is we made this decision because, in the future, merge_group_id values might not be project id's, but some other indicator variable (from a future metadata file) to group, for example, all samples of diagnosis X, which has the potential to include libraries from multiple projects.

This issue is meant to facilitate some discussion about this point. Is this still our goal to keep this flexibility? If not, we might consider changing the publish directory to the project directory itself, not within merged/, which may make life a little easier on the dev side.

Also worth noting these external instructions, where we tell users that their merge group is expected to be the same as their project id:
https://github.com/AlexsLemonade/scpca-nf/blob/development/external-instructions.md#the-mergenf-workflow

[jashapiro]

I would want to keep this flexibility, yes. But we might be able to do this by reversing the nesting order:

 publishDir "${params.results_dir}/${merge_group_id}/merged" 

This would nest the files within the ProjectID when it exists, and create new directories when it does not.

[sjspielman]

I would want to keep this flexibility, yes. But we might be able to do this by reversing the nesting order:

This is a nice option!

[sjspielman]

We've decided to implement this as @jashapiro suggested in his comment above.

CC @davidsmejia @arkid15r - after I implement this, I'll do another test data release.

[sjspielman]

Closed by #706