samtools sort

[simonvh]

seq2science/seq2science/rules/alignment.smk

Line 302 in cf045b3

samtools sort {params.sort_order} -@ {params.threads} {input} -o {output} -T {params.out_dir}/{wildcards.assembly}-{wildcards.sample}.tmp 2> {log}

Have you tried / benchmarked setting larger memory for samtools sort (-m 4G)? With the defautl settings samtools sort generates many small files for sorting. I think this can slow down the process. An alternative is also sambamba which seems to sort faster (at least, the last time I looked at it).

[Maarten-vd-Sande]

Yes I am aware. However I think the issue is that some assemblies consist of many many many contigs, and that blows up in memory. But that is from memory ( +/- a year ago). I will leave this open, and test again.

You can already use sambamba as a sorter, but ... ofcourse there are minor differences in the way they sort. Samtools & sambamba sort slightly different on read names (not coordinates, and not sure anymore on what, but think about order of capital letters), which makes some tools very unhappy.

So I decided that every downstream tool MUST use the samtools-coordinate sorted bam, since I never had troubles with those. If you only do alignment, then you can choose the sort_order (coordinate/queryname) and sorter (samtools/sambamba).

[Maarten-vd-Sande]

https://vanheeringen-lab.github.io/seq2science/content/schemas.html#alignment-workflow-specific

https://github.com/vanheeringen-lab/seq2science/blob/master/seq2science/workflows/alignment/config.yaml#L21

[simonvh]

Ah, sounds reasonable. I suspected there might be a good reason. In any case, it might be good to have the -m option of samtools configurable.

[Maarten-vd-Sande]

Note for instance that genrich specifically uses sambamba-queryname sorted bams, this is because it explodes when you give it samtools-queryname sorted bams. This means that during some workflows two versions of the same bam exist, but they differ just by sorters/sort orders. See for instance: biod/sambamba#132.

Ah, sounds reasonable. I suspected there might be a good reason. In any case, it might be good to have the -m option of samtools configurable.

Yes, will do