call mutation from multiple regions and multiple chromosomes

[WeiweiBian]

Hi, I am trying to call some targeted mutations from multiple targeted regions and chromosomes by Deepvariant. I followed the manual of make_examples.py to separate the multiple regions by space(

deepvariant/deepvariant/make_examples.py

Line 116 in aff131a

'Optional. Space-separated list of regions we want to process. Elements '

), but it showed an error like the following:
sudo docker run -v "${INPUT_DIR}":"/input" -v "${OUTPUT_DIR}:/output" google/deepvariant:"${BIN_VERSION}" /opt/deepvariant/bin/run_deepvariant --model_type=WGS --ref=/input/Homo_sapiens_assembly19.fasta --reads=/input/proper.bam
--regions="3:178936057-178936106 3:178952054-178952106"
--output_vcf=/output/outputch3.vcf.gz --output_gvcf=/output/outputchr3.g.vcf.gz
--num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004
E0416 22:33:24.439428 140413429749504 errors.py:61] Command line parsing failure: make_examples does not accept positional arguments but some are present on the command line: "['/tmp/Bazel.runfiles_xs3AuO/runfiles/com_google_deepvariant/deepvariant/make_examples.py', '3:178952054-178952106']".
I wonder whether this issue is due to the chromosome index of my sample is 1-22 without ‘chr’ and how to run it with multiple regions and multiple chromosomes.

Another issue is that when I changed the region to the bed file which includes all targeted regions, the variants called are all false-positive and none true variants included.

Furthermore, my sample is from deep sequencing on targeted regions. I have followed issue #62 to change the pileup_image_hight and downsample_fraction options, but the results are not improved.

Thanks a lot!
Best regards,
Weiwei

[MariaNattestad]

Hi Weiwei

It says make_examples does not accept positional arguments but some are present on the command line because it is getting some arguments that don't have a -- parameter name in front of them. It looks like you need to put --regions before the locus: --regions 3:178952054-178952106. If that doesn't work, can you share the command you used?

[WeiweiBian]

Thanks for your reply, but it didn't work. If I type --regions "3:178936057-178936106" --regions "3:178952054-178952106", then the deepvariant only accepts the second region as an input. I have also tried several ways, such as --regions="3:178936057-178936106" "3:178952054-178952106" , --regions="3:178936057-178936106 3:178952054-178952106", --regions=3:178936057-178936106 3:178952054-178952106, --regions=3:178936057-178936106 178952054-178952106, --regions "3:178936057-178936106 178952054-178952106", --regions "3:178936057-178936106,178952054-178952106", and none of them works. The log is attached here. Thanks a lot!
problemetic logs with 'regions'.txt

[MariaNattestad]

Oh, sorry I couldn't see that you already had one --regions in there.
I believe this one below (taken from your attached logs) should have worked, and it works for me locally, so I'll leave this here and ask my colleagues to weigh in.

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

E0416 22:39:09.366390 140299630700288 errors.py:61] Command line parsing failure: make_examples does not accept positional arguments but some are present on the command line: "['/tmp/Bazel.runfiles_WCb7xE/runfiles/com_google_deepvariant/deepvariant/make_examples.py', '3:178952054-178952106']".
``

[MariaNattestad]

I was able to reproduce the issue you noticed when running the docker build. The issue looks to be that the quotes are used up and not passed on to the make_examples command intact. I was able to get it working by doing this: --regions="'3:178936057-178936106 3:178952054-178952106'". This uses single quotes encased by double quotes, while the other way around did not work for some reason. Does that work for you?

Thanks for bringing this up. I will make a note of this, so we can find a fix or at least update the documentation.

[WeiweiBian]

Thank a lot for your reply. The 'single quotes encased by double quotes' works for this case, but when I added more regions, it couldn't give me an available output and showed an exit status 247.

[MariaNattestad]

Interesting. Could you share the command you ran and the error message?

…

On Tue, Apr 28, 2020 at 1:45 PM WeiweiBian ***@***.***> wrote: Thank a lot for your reply. The 'single quotes encased by double quotes' works for this case, but when I added more regions, it couldn't give me an available output and showed an exit status 247. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#305 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4W4PIOBHJ5DAXPKER4A63RO457FANCNFSM4MQ4ZT2Q> .

[WeiweiBian]

Hi, thanks for your reply. When I retried it, it showed the error with 'subprocess.CalledProcessError: Command 'time seq 0 0 | parallel --halt 2 --line-buffer /opt/deepvariant/bin/make_examples --mode calling' and --task {}' returned non-zero exit status 1.
For the codes I used, is it available to send to your email? Thanks!

[MariaNattestad]

There should be a longer error trace coming from DeepVariant itself, which will be more informative than just the exit status code. Can you share that? If you need to, you can email me at marianattestad@google.com.

[MariaNattestad]

I'm going to close this since given our follow-up over email, the original issue with --regions is no longer the problem.

For anyone else reading this, --regions with more than 2 regions does work, and putting extra quotes exactly like this: --regions="'3:178936057-178936106 3:178952054-178952106'" is the way to ensure that the quotes encasing your space-separated regions don't disappear between your docker run command and actually getting that argument passed into make_examples.