Input file movement (although read-only) .bam/.bai, .fa/.fai

[dansteiert]

Hey there, I am running this via apptainer on a local machien.

apptainer run --mount type=bind,src=MY/PATH/IN,dst=/home/dnanexus/in --mount type=bind,src=MY/PATH/OUT,dst=/home/dnanexus/out parliament2_latest.sif --bam /home/dnanexus/in/SUBFOLDER/FILE.bam --bai /home/dnanexus/in/SUBFOLDER/FILE.bai --prefix PREFIX --ref_genome /home/dnanexus/in/SUBFOLDER/FILE.fa --fai /home/dnanexus/in/SUBFOLDER/FILE.fa.fai --breakdancer --breakseq --manta --cnvnator --lumpy --genotype --svviz --svviz_only_validated_candidates

What no happens:
Loads of folders appear in MY/PATH/IN and some files, amongst others my bam file (a couple GB big files get moved)
This is the case for my .bam, .bam.bai -> these file are actually moved and thus make my folder structure redundant and could actually lead to removing the bam file entierly, which could have severe consequences!! (at least in terms of computational time).

The reference fasta is actually copied, which is not necessarily necessary, but at least it is not moved!

This actually happend to me - i deleted the "noice" including my bam file. afterwards i recovered it and made sure again that it is read only. Apparently, this is overwritten (because I am owner?) and the files are moved anyway.

A look into the logs:

  2 ln: failed to create symbolic link './input.bam': File exists
  3 ln: failed to create symbolic link './input.bam.bai': File exists
  4 Processing bam input.bam
  5 [bam_header_read] EOF marker is absent. The input is probably truncated.
  6 [bam_header_read] invalid BAM binary header (this is not a BAM file).
  7 Failed to open samfile input.bam

Two issues arise -> what you would think should happen, the symbolic link is created -> which does not happen because the file is already moved apparently -> made sure by clearing the files beforehand!

And the next issue is that my bam file cannot be read.
i checked from within the container with samtools quickcheck /MY/PATH/IN/SUBFOLDER/FILE.bam if this is the case and no, it looks as usual - perfectly fine!

My suggestions for usage outside the dnanexus:

• no arbitrary moving of files
• provide an option to mount temporary directories for these "noice" outputs
• since symlinks from outside do not work, allow for two input mounts bam/bai and another for the fa/fai

About my bam read issue - this might be unrelated, but still interesting since I used the internal samtools to validate it for the EOF block -> so maybe another program checks which does not agree with it.
Anyway Best!