C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions "3:178936057-178936106" --regions "3:178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

I0415 14:14:04.169197 140127625611008 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0415 14:14:04.184864 140127625611008 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0415 14:14:04.185085 140127625611008 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0415 14:14:04.187787 140127625611008 make_examples.py:1324] Preparing inputs
I0415 14:14:04.218621 140127625611008 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0415 14:14:04.231635 140127625611008 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
I0415 14:14:04.242830 140127625611008 make_examples.py:1330] Writing examples to /tmp/deepvariant_tmp_output/make_examples.tfrecord-00000-of-00001.gz
I0415 14:14:04.243356 140127625611008 make_examples.py:1334] Writing gvcf records to /tmp/deepvariant_tmp_output/gvcf.tfrecord-00000-of-00001.gz
I0415 14:14:04.251851 140127625611008 make_examples.py:905] Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
2020-04-15 14:14:04.254247: I third_party/nucleus/io/sam_reader.cc:660] Setting HTS_OPT_BLOCK_SIZE to 134217728
I0415 14:14:04.731903 140127625611008 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0415 14:14:04.763411 140127625611008 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0415 14:14:06.243246 140127625611008 make_examples.py:1363] Task 0: 2 candidates (2 examples) [2.00s elapsed]
I0415 14:14:06.245687 140127625611008 make_examples.py:1380] Found 2 candidate variants
I0415 14:14:06.245980 140127625611008 make_examples.py:1381] Created 2 examples

real  0m3.849s
user  0m2.897s
sys 0m0.740s

***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --checkpoint "/opt/models/wgs/model.ckpt"

I0415 14:14:08.232855 140318873716480 call_variants.py:313] Set KMP_BLOCKTIME to 0
2020-04-15 14:14:08.246479: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-15 14:14:08.251482: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2496000000 Hz
2020-04-15 14:14:08.251657: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4623690 executing computations on platform Host. Devices:
2020-04-15 14:14:08.251700: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-04-15 14:14:08.252736: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
I0415 14:14:08.265214 140318873716480 modeling.py:560] Initializing model with random parameters
W0415 14:14:08.267050 140318873716480 estimator.py:1760] Using temporary folder as model directory: /tmp/tmpvtiRrX
I0415 14:14:08.267796 140318873716480 estimator.py:201] Using config: {'_save_checkpoints_secs': 600, '_session_config': , '_keep_checkpoint_max': 100000, '_task_type': 'worker', '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f9e50f17190>, '_model_dir': '/tmp/tmpvtiRrX', '_protocol': None, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_save_summary_steps': 100, '_device_fn': None, '_experimental_distribute': None, '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_evaluation_master': '', '_eval_distribute': None, '_global_id_in_cluster': 0, '_master': ''}
I0415 14:14:08.268425 140318873716480 call_variants.py:381] Writing calls to /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
W0415 14:14:08.273626 140318873716480 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
I0415 14:14:08.285984 140318873716480 data_providers.py:367] self.input_read_threads=8
W0415 14:14:08.286432 140318873716480 deprecation.py:323] From /tmp/Bazel.runfiles_DKn1ah/runfiles/com_google_deepvariant/deepvariant/data_providers.py:372: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
I0415 14:14:08.303431 140318873716480 data_providers.py:373] self.input_map_threads=48
I0415 14:14:08.321943 140318873716480 estimator.py:1111] Calling model_fn.
W0415 14:14:08.322134 140318873716480 deprecation.py:323] From /tmp/Bazel.runfiles_DKn1ah/runfiles/com_google_deepvariant/deepvariant/modeling.py:880: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0415 14:14:08.324070 140318873716480 deprecation.py:323] From /tmp/Bazel.runfiles_DKn1ah/runfiles/com_google_deepvariant/deepvariant/modeling.py:882: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
I0415 14:14:10.836234 140318873716480 estimator.py:1113] Done calling model_fn.
I0415 14:14:11.982916 140318873716480 monitored_session.py:222] Graph was finalized.
W0415 14:14:11.983591 140318873716480 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
I0415 14:14:11.984663 140318873716480 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0415 14:14:13.178937 140318873716480 session_manager.py:491] Running local_init_op.
I0415 14:14:13.234967 140318873716480 session_manager.py:493] Done running local_init_op.
I0415 14:14:13.648705 140318873716480 modeling.py:410] Reloading EMA...
I0415 14:14:13.649909 140318873716480 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0415 14:14:15.975042 140318873716480 call_variants.py:399] Processed 1 examples in 1 batches [770.549 sec per 100]
I0415 14:14:16.008719 140318873716480 call_variants.py:401] Done evaluating variants

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.


***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "/input/Homo_sapiens_assembly19.fasta" --infile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --outfile "/output/outputch33.vcf.gz" --nonvariant_site_tfrecord_path "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --gvcf_outfile "/output/outputchr33.g.vcf.gz"


real  0m9.871s
user  0m9.287s
sys 0m0.812s
2020-04-15 14:14:17.813470: I deepvariant/postprocess_variants.cc:88] Read from: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
2020-04-15 14:14:17.813718: I deepvariant/postprocess_variants.cc:97] Done reading: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz. #entries in single_site_calls = 2
2020-04-15 14:14:17.813793: I deepvariant/postprocess_variants.cc:101] Total #entries in single_site_calls = 2
2020-04-15 14:14:17.813881: I deepvariant/postprocess_variants.cc:103] Start SortSingleSiteCalls
2020-04-15 14:14:17.813920: I deepvariant/postprocess_variants.cc:105] Done SortSingleSiteCalls
I0415 14:14:17.814696 140294581552896 postprocess_variants.py:972] CVO sorting took 2.50180562337e-05 minutes
I0415 14:14:17.815399 140294581552896 postprocess_variants.py:974] Transforming call_variants_output to variants.
I0415 14:14:17.816286 140294581552896 postprocess_variants.py:1000] Merging and writing variants to VCF and gVCF.
I0415 14:14:17.830699 140294581552896 genomics_writer.py:172] Writing /output/outputch33.vcf.gz with NativeVcfWriter
I0415 14:14:17.838656 140294581552896 genomics_writer.py:172] Writing /output/outputchr33.g.vcf.gz with NativeVcfWriter
I0415 14:14:18.033109 140294581552896 postprocess_variants.py:1020] Finished writing VCF and gVCF in 0.00361315011978 minutes.
I0415 14:14:18.045304 140294581552896 genomics_reader.py:223] Reading /output/outputch33.vcf.gz with NativeVcfReader

real  0m2.210s
user  0m1.832s
sys 0m0.484s

C8-C02W70QBHV2H:Lib002 weibia$ gzcat ../../dnn-output1/outputch33.vcf.gz | grep -v '^##' 
#CHROM  POS ID  REF ALT QUAL  FILTER  INFO  FORMAT  default
3 178952060 . T C 0 RefCall . GT:GQ:DP:AD:VAF:PL  0/0:31:1363:1355,6:0.00440205:0,32,39
3 178952085 . A G 2.5 RefCall . GT:GQ:DP:AD:VAF:PL  ./.:4:1350:1201,148:0.10963:0,1,28


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions="3:178936057-178936106" "3:178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106" --vsc_min_fraction_snps "0.004" --task {}

I0416 22:31:52.827272 139854852523776 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0416 22:31:52.848270 139854852523776 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0416 22:31:52.848469 139854852523776 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0416 22:31:52.851408 139854852523776 make_examples.py:1324] Preparing inputs
I0416 22:31:52.917735 139854852523776 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:31:52.931478 139854852523776 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
I0416 22:31:52.940166 139854852523776 make_examples.py:1330] Writing examples to /tmp/deepvariant_tmp_output/make_examples.tfrecord-00000-of-00001.gz
I0416 22:31:52.940557 139854852523776 make_examples.py:1334] Writing gvcf records to /tmp/deepvariant_tmp_output/gvcf.tfrecord-00000-of-00001.gz
I0416 22:31:52.947566 139854852523776 make_examples.py:905] Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
2020-04-16 22:31:52.949447: I third_party/nucleus/io/sam_reader.cc:660] Setting HTS_OPT_BLOCK_SIZE to 134217728
I0416 22:31:53.449624 139854852523776 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:31:53.481045 139854852523776 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:31:54.444595 139854852523776 make_examples.py:1363] Task 0: 2 candidates (2 examples) [1.50s elapsed]
I0416 22:31:54.446660 139854852523776 make_examples.py:1380] Found 2 candidate variants
I0416 22:31:54.447066 139854852523776 make_examples.py:1381] Created 2 examples

real	0m3.469s
user	0m2.448s
sys	0m0.786s

***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --checkpoint "/opt/models/wgs/model.ckpt"

I0416 22:31:56.495668 139637324416768 call_variants.py:313] Set KMP_BLOCKTIME to 0
2020-04-16 22:31:56.510024: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-16 22:31:56.515093: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2496000000 Hz
2020-04-16 22:31:56.515273: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4d947d0 executing computations on platform Host. Devices:
2020-04-16 22:31:56.516414: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-04-16 22:31:56.517531: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
I0416 22:31:56.530441 139637324416768 modeling.py:560] Initializing model with random parameters
W0416 22:31:56.533519 139637324416768 estimator.py:1760] Using temporary folder as model directory: /tmp/tmphexnaQ
I0416 22:31:56.534876 139637324416768 estimator.py:201] Using config: {'_save_checkpoints_secs': 600, '_session_config': , '_keep_checkpoint_max': 100000, '_task_type': 'worker', '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7effa1707190>, '_model_dir': '/tmp/tmphexnaQ', '_protocol': None, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_save_summary_steps': 100, '_device_fn': None, '_experimental_distribute': None, '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_evaluation_master': '', '_eval_distribute': None, '_global_id_in_cluster': 0, '_master': ''}
I0416 22:31:56.535871 139637324416768 call_variants.py:381] Writing calls to /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
W0416 22:31:56.540359 139637324416768 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
I0416 22:31:56.551502 139637324416768 data_providers.py:367] self.input_read_threads=8
W0416 22:31:56.552205 139637324416768 deprecation.py:323] From /tmp/Bazel.runfiles_z8m8Qe/runfiles/com_google_deepvariant/deepvariant/data_providers.py:372: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
I0416 22:31:56.571508 139637324416768 data_providers.py:373] self.input_map_threads=48
I0416 22:31:56.601591 139637324416768 estimator.py:1111] Calling model_fn.
W0416 22:31:56.602086 139637324416768 deprecation.py:323] From /tmp/Bazel.runfiles_z8m8Qe/runfiles/com_google_deepvariant/deepvariant/modeling.py:880: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0416 22:31:56.604737 139637324416768 deprecation.py:323] From /tmp/Bazel.runfiles_z8m8Qe/runfiles/com_google_deepvariant/deepvariant/modeling.py:882: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
I0416 22:31:59.028340 139637324416768 estimator.py:1113] Done calling model_fn.
I0416 22:32:00.203913 139637324416768 monitored_session.py:222] Graph was finalized.
W0416 22:32:00.204875 139637324416768 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
I0416 22:32:00.206335 139637324416768 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:32:01.210071 139637324416768 session_manager.py:491] Running local_init_op.
I0416 22:32:01.260570 139637324416768 session_manager.py:493] Done running local_init_op.
I0416 22:32:01.648067 139637324416768 modeling.py:410] Reloading EMA...
I0416 22:32:01.649200 139637324416768 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:32:03.936305 139637324416768 call_variants.py:399] Processed 1 examples in 1 batches [739.974 sec per 100]
I0416 22:32:03.971971 139637324416768 call_variants.py:401] Done evaluating variants

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.


real	0m9.612s
user	0m8.974s
sys	0m0.842s

***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "/input/Homo_sapiens_assembly19.fasta" --infile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --outfile "/output/outputch33.vcf.gz" --nonvariant_site_tfrecord_path "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --gvcf_outfile "/output/outputchr33.g.vcf.gz"

2020-04-16 22:32:05.760942: I deepvariant/postprocess_variants.cc:88] Read from: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
2020-04-16 22:32:05.761188: I deepvariant/postprocess_variants.cc:97] Done reading: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz. #entries in single_site_calls = 2
2020-04-16 22:32:05.761520: I deepvariant/postprocess_variants.cc:101] Total #entries in single_site_calls = 2
2020-04-16 22:32:05.761755: I deepvariant/postprocess_variants.cc:103] Start SortSingleSiteCalls
2020-04-16 22:32:05.762115: I deepvariant/postprocess_variants.cc:105] Done SortSingleSiteCalls
I0416 22:32:05.762708 140328117516032 postprocess_variants.py:972] CVO sorting took 3.37481498718e-05 minutes
I0416 22:32:05.763559 140328117516032 postprocess_variants.py:974] Transforming call_variants_output to variants.
I0416 22:32:05.764291 140328117516032 postprocess_variants.py:1000] Merging and writing variants to VCF and gVCF.
I0416 22:32:05.773344 140328117516032 genomics_writer.py:172] Writing /output/outputch33.vcf.gz with NativeVcfWriter
I0416 22:32:05.776874 140328117516032 genomics_writer.py:172] Writing /output/outputchr33.g.vcf.gz with NativeVcfWriter
I0416 22:32:05.986813 140328117516032 postprocess_variants.py:1020] Finished writing VCF and gVCF in 0.00370803276698 minutes.
I0416 22:32:05.999376 140328117516032 genomics_reader.py:223] Reading /output/outputch33.vcf.gz with NativeVcfReader

real	0m2.211s
user	0m1.818s
sys	0m0.491s
C8-C02W70QBHV2H:Lib002 weibia$ gzcat ../../dnn-output1/outputch33.vcf.gz | grep -v '^##' |column -t
#CHROM  POS        ID  REF  ALT  QUAL  FILTER   INFO  FORMAT              default
3       178936085  .   A    G    0     RefCall  .     GT:GQ:DP:AD:VAF:PL  0/0:32:1428:1421,6:0.00420168:0,33,36
3       178936091  .   G    A    0     RefCall  .     GT:GQ:DP:AD:VAF:PL  0/0:32:1446:1433,9:0.00622407:0,32,37


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions="3:178936057-178936106 3:178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

E0416 22:33:24.439428 140413429749504 errors.py:61] Command line parsing failure: make_examples does not accept positional arguments but some are present on the command line: "['/tmp/Bazel.runfiles_xs3AuO/runfiles/com_google_deepvariant/deepvariant/make_examples.py', '3:178952054-178952106']".

real	0m1.789s
user	0m1.452s
sys	0m0.539s
Traceback (most recent call last):
  File "/opt/deepvariant/bin/run_deepvariant.py", line 317, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/deepvariant/bin/run_deepvariant.py", line 307, in main
    subprocess.check_call(command, shell=True, executable='/bin/bash')
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}' returned non-zero exit status 1


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions=3:178936057-178936106 3:178952054-178952106  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004


***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106" --vsc_min_fraction_snps "0.004" --task {}

I0416 22:35:07.361725 139788149860096 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0416 22:35:07.373991 139788149860096 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0416 22:35:07.374423 139788149860096 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0416 22:35:07.377644 139788149860096 make_examples.py:1324] Preparing inputs
I0416 22:35:07.407205 139788149860096 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:07.420622 139788149860096 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
I0416 22:35:07.429265 139788149860096 make_examples.py:1330] Writing examples to /tmp/deepvariant_tmp_output/make_examples.tfrecord-00000-of-00001.gz
I0416 22:35:07.429605 139788149860096 make_examples.py:1334] Writing gvcf records to /tmp/deepvariant_tmp_output/gvcf.tfrecord-00000-of-00001.gz
I0416 22:35:07.437205 139788149860096 make_examples.py:905] Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
2020-04-16 22:35:07.439070: I third_party/nucleus/io/sam_reader.cc:660] Setting HTS_OPT_BLOCK_SIZE to 134217728
I0416 22:35:07.914983 139788149860096 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:07.948997 139788149860096 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:08.896856 139788149860096 make_examples.py:1363] Task 0: 2 candidates (2 examples) [1.47s elapsed]
I0416 22:35:08.898838 139788149860096 make_examples.py:1380] Found 2 candidate variants
I0416 22:35:08.899275 139788149860096 make_examples.py:1381] Created 2 examples

real	0m3.352s
user	0m2.389s
sys	0m0.745s

***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --checkpoint "/opt/models/wgs/model.ckpt"

I0416 22:35:10.948266 140663709587200 call_variants.py:313] Set KMP_BLOCKTIME to 0
2020-04-16 22:35:10.962783: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-16 22:35:10.967805: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2496000000 Hz
2020-04-16 22:35:10.968289: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4d2cc90 executing computations on platform Host. Devices:
2020-04-16 22:35:10.968336: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-04-16 22:35:10.968969: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
I0416 22:35:10.981944 140663709587200 modeling.py:560] Initializing model with random parameters
W0416 22:35:10.983797 140663709587200 estimator.py:1760] Using temporary folder as model directory: /tmp/tmpXs0_NE
I0416 22:35:10.985256 140663709587200 estimator.py:201] Using config: {'_save_checkpoints_secs': 600, '_session_config': , '_keep_checkpoint_max': 100000, '_task_type': 'worker', '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fee9ac34190>, '_model_dir': '/tmp/tmpXs0_NE', '_protocol': None, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_save_summary_steps': 100, '_device_fn': None, '_experimental_distribute': None, '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_evaluation_master': '', '_eval_distribute': None, '_global_id_in_cluster': 0, '_master': ''}
I0416 22:35:10.986727 140663709587200 call_variants.py:381] Writing calls to /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
W0416 22:35:10.991564 140663709587200 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
I0416 22:35:11.007857 140663709587200 data_providers.py:367] self.input_read_threads=8
W0416 22:35:11.008382 140663709587200 deprecation.py:323] From /tmp/Bazel.runfiles_tvJiG4/runfiles/com_google_deepvariant/deepvariant/data_providers.py:372: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
I0416 22:35:11.022855 140663709587200 data_providers.py:373] self.input_map_threads=48
I0416 22:35:11.042190 140663709587200 estimator.py:1111] Calling model_fn.
W0416 22:35:11.042809 140663709587200 deprecation.py:323] From /tmp/Bazel.runfiles_tvJiG4/runfiles/com_google_deepvariant/deepvariant/modeling.py:880: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0416 22:35:11.045088 140663709587200 deprecation.py:323] From /tmp/Bazel.runfiles_tvJiG4/runfiles/com_google_deepvariant/deepvariant/modeling.py:882: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
I0416 22:35:13.479180 140663709587200 estimator.py:1113] Done calling model_fn.
I0416 22:35:14.646814 140663709587200 monitored_session.py:222] Graph was finalized.
W0416 22:35:14.647921 140663709587200 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
I0416 22:35:14.649327 140663709587200 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:35:15.641863 140663709587200 session_manager.py:491] Running local_init_op.
I0416 22:35:15.692276 140663709587200 session_manager.py:493] Done running local_init_op.
I0416 22:35:16.090616 140663709587200 modeling.py:410] Reloading EMA...
I0416 22:35:16.091758 140663709587200 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:35:18.406459 140663709587200 call_variants.py:399] Processed 1 examples in 1 batches [741.869 sec per 100]
I0416 22:35:18.442141 140663709587200 call_variants.py:401] Done evaluating variants

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.


real	0m9.624s
user	0m8.990s
sys	0m0.825s

***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "/input/Homo_sapiens_assembly19.fasta" --infile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --outfile "/output/outputch33.vcf.gz" --nonvariant_site_tfrecord_path "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --gvcf_outfile "/output/outputchr33.g.vcf.gz"

2020-04-16 22:35:20.226652: I deepvariant/postprocess_variants.cc:88] Read from: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
2020-04-16 22:35:20.226970: I deepvariant/postprocess_variants.cc:97] Done reading: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz. #entries in single_site_calls = 2
2020-04-16 22:35:20.227118: I deepvariant/postprocess_variants.cc:101] Total #entries in single_site_calls = 2
2020-04-16 22:35:20.227243: I deepvariant/postprocess_variants.cc:103] Start SortSingleSiteCalls
2020-04-16 22:35:20.227334: I deepvariant/postprocess_variants.cc:105] Done SortSingleSiteCalls
I0416 22:35:20.227951 139999670544128 postprocess_variants.py:972] CVO sorting took 2.31345494588e-05 minutes
I0416 22:35:20.228981 139999670544128 postprocess_variants.py:974] Transforming call_variants_output to variants.
I0416 22:35:20.229939 139999670544128 postprocess_variants.py:1000] Merging and writing variants to VCF and gVCF.
I0416 22:35:20.232992 139999670544128 genomics_writer.py:172] Writing /output/outputch33.vcf.gz with NativeVcfWriter
I0416 22:35:20.236294 139999670544128 genomics_writer.py:172] Writing /output/outputchr33.g.vcf.gz with NativeVcfWriter
I0416 22:35:20.443344 139999670544128 postprocess_variants.py:1020] Finished writing VCF and gVCF in 0.00355611642202 minutes.
I0416 22:35:20.455368 139999670544128 genomics_reader.py:223] Reading /output/outputch33.vcf.gz with NativeVcfReader

real	0m2.193s
user	0m1.790s
sys	0m0.518s
C8-C02W70QBHV2H:Lib002 weibia$ gzcat ../../dnn-output1/outputch33.vcf.gz | grep -v '^##' |column -t
#CHROM  POS        ID  REF  ALT  QUAL  FILTER   INFO  FORMAT              default
3       178936085  .   A    G    0     RefCall  .     GT:GQ:DP:AD:VAF:PL  0/0:32:1428:1421,6:0.00420168:0,33,36
3       178936091  .   G    A    0     RefCall  .     GT:GQ:DP:AD:VAF:PL  0/0:32:1446:1433,9:0.00622407:0,32,37


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions=3:178936057-178936106 178952054-178952106  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004


***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106" --vsc_min_fraction_snps "0.004" --task {}

I0416 22:35:55.981638 140163970127616 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0416 22:35:55.993271 140163970127616 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0416 22:35:55.993468 140163970127616 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0416 22:35:55.996074 140163970127616 make_examples.py:1324] Preparing inputs
I0416 22:35:56.013432 140163970127616 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:56.025870 140163970127616 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
I0416 22:35:56.034486 140163970127616 make_examples.py:1330] Writing examples to /tmp/deepvariant_tmp_output/make_examples.tfrecord-00000-of-00001.gz
I0416 22:35:56.034854 140163970127616 make_examples.py:1334] Writing gvcf records to /tmp/deepvariant_tmp_output/gvcf.tfrecord-00000-of-00001.gz
I0416 22:35:56.041254 140163970127616 make_examples.py:905] Starting from v0.9.0, --use_ref_for_cram is default to true. If you are using CRAM input, note that we will decode CRAM using the reference you passed in with --ref
2020-04-16 22:35:56.043126: I third_party/nucleus/io/sam_reader.cc:660] Setting HTS_OPT_BLOCK_SIZE to 134217728
I0416 22:35:56.503689 140163970127616 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:56.536153 140163970127616 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0416 22:35:57.469511 140163970127616 make_examples.py:1363] Task 0: 2 candidates (2 examples) [1.43s elapsed]
I0416 22:35:57.471461 140163970127616 make_examples.py:1380] Found 2 candidate variants
I0416 22:35:57.471781 140163970127616 make_examples.py:1381] Created 2 examples

real	0m3.311s
user	0m2.348s
sys	0m0.768s

***** Running the command:*****
time /opt/deepvariant/bin/call_variants --outfile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --checkpoint "/opt/models/wgs/model.ckpt"

I0416 22:35:59.484941 139762471368448 call_variants.py:313] Set KMP_BLOCKTIME to 0
2020-04-16 22:35:59.499843: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-16 22:35:59.505010: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2496000000 Hz
2020-04-16 22:35:59.505184: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x5079b20 executing computations on platform Host. Devices:
2020-04-16 22:35:59.505201: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-04-16 22:35:59.506045: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
I0416 22:35:59.518352 139762471368448 modeling.py:560] Initializing model with random parameters
W0416 22:35:59.520221 139762471368448 estimator.py:1760] Using temporary folder as model directory: /tmp/tmpy193So
I0416 22:35:59.521051 139762471368448 estimator.py:201] Using config: {'_save_checkpoints_secs': 600, '_session_config': , '_keep_checkpoint_max': 100000, '_task_type': 'worker', '_train_distribute': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f1cc4c76190>, '_model_dir': '/tmp/tmpy193So', '_protocol': None, '_save_checkpoints_steps': None, '_keep_checkpoint_every_n_hours': 10000, '_service': None, '_num_ps_replicas': 0, '_tf_random_seed': None, '_save_summary_steps': 100, '_device_fn': None, '_experimental_distribute': None, '_num_worker_replicas': 1, '_task_id': 0, '_log_step_count_steps': 100, '_evaluation_master': '', '_eval_distribute': None, '_global_id_in_cluster': 0, '_master': ''}
I0416 22:35:59.522115 139762471368448 call_variants.py:381] Writing calls to /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
W0416 22:35:59.527087 139762471368448 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
I0416 22:35:59.539494 139762471368448 data_providers.py:367] self.input_read_threads=8
W0416 22:35:59.539951 139762471368448 deprecation.py:323] From /tmp/Bazel.runfiles_AoA3fe/runfiles/com_google_deepvariant/deepvariant/data_providers.py:372: parallel_interleave (from tensorflow.contrib.data.python.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.experimental.parallel_interleave(...)`.
I0416 22:35:59.558039 139762471368448 data_providers.py:373] self.input_map_threads=48
I0416 22:35:59.578537 139762471368448 estimator.py:1111] Calling model_fn.
W0416 22:35:59.578888 139762471368448 deprecation.py:323] From /tmp/Bazel.runfiles_AoA3fe/runfiles/com_google_deepvariant/deepvariant/modeling.py:880: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0416 22:35:59.585143 139762471368448 deprecation.py:323] From /tmp/Bazel.runfiles_AoA3fe/runfiles/com_google_deepvariant/deepvariant/modeling.py:882: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
I0416 22:36:02.015083 139762471368448 estimator.py:1113] Done calling model_fn.
I0416 22:36:03.171756 139762471368448 monitored_session.py:222] Graph was finalized.
W0416 22:36:03.172646 139762471368448 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
I0416 22:36:03.173840 139762471368448 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:36:04.185723 139762471368448 session_manager.py:491] Running local_init_op.
I0416 22:36:04.236701 139762471368448 session_manager.py:493] Done running local_init_op.
I0416 22:36:04.623821 139762471368448 modeling.py:410] Reloading EMA...
I0416 22:36:04.624958 139762471368448 saver.py:1270] Restoring parameters from /opt/models/wgs/model.ckpt
I0416 22:36:06.882502 139762471368448 call_variants.py:399] Processed 1 examples in 1 batches [735.903 sec per 100]
I0416 22:36:06.919326 139762471368448 call_variants.py:401] Done evaluating variants

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.


real	0m9.534s
user	0m8.945s
sys	0m0.790s

***** Running the command:*****
time /opt/deepvariant/bin/postprocess_variants --ref "/input/Homo_sapiens_assembly19.fasta" --infile "/tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz" --outfile "/output/outputch33.vcf.gz" --nonvariant_site_tfrecord_path "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --gvcf_outfile "/output/outputchr33.g.vcf.gz"

2020-04-16 22:36:08.721411: I deepvariant/postprocess_variants.cc:88] Read from: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz
2020-04-16 22:36:08.721804: I deepvariant/postprocess_variants.cc:97] Done reading: /tmp/deepvariant_tmp_output/call_variants_output.tfrecord.gz. #entries in single_site_calls = 2
2020-04-16 22:36:08.722133: I deepvariant/postprocess_variants.cc:101] Total #entries in single_site_calls = 2
2020-04-16 22:36:08.722209: I deepvariant/postprocess_variants.cc:103] Start SortSingleSiteCalls
2020-04-16 22:36:08.722686: I deepvariant/postprocess_variants.cc:105] Done SortSingleSiteCalls
I0416 22:36:08.723577 139690135930624 postprocess_variants.py:972] CVO sorting took 3.88503074646e-05 minutes
I0416 22:36:08.724469 139690135930624 postprocess_variants.py:974] Transforming call_variants_output to variants.
I0416 22:36:08.725245 139690135930624 postprocess_variants.py:1000] Merging and writing variants to VCF and gVCF.
I0416 22:36:08.728660 139690135930624 genomics_writer.py:172] Writing /output/outputch33.vcf.gz with NativeVcfWriter
I0416 22:36:08.731618 139690135930624 genomics_writer.py:172] Writing /output/outputchr33.g.vcf.gz with NativeVcfWriter
I0416 22:36:08.918975 139690135930624 postprocess_variants.py:1020] Finished writing VCF and gVCF in 0.00322891871134 minutes.
I0416 22:36:08.932770 139690135930624 genomics_reader.py:223] Reading /output/outputch33.vcf.gz with NativeVcfReader

real	0m2.191s
user	0m1.816s
sys	0m0.496s


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions "3:178936057-178936106 178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

E0416 22:38:23.320422 140500462393088 errors.py:61] Command line parsing failure: make_examples does not accept positional arguments but some are present on the command line: "['/tmp/Bazel.runfiles_YZMD8b/runfiles/com_google_deepvariant/deepvariant/make_examples.py', '178952054-178952106']".

real	0m1.770s
user	0m1.452s
sys	0m0.530s
Traceback (most recent call last):
  File "/opt/deepvariant/bin/run_deepvariant.py", line 317, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/deepvariant/bin/run_deepvariant.py", line 307, in main
    subprocess.check_call(command, shell=True, executable='/bin/bash')
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}' returned non-zero exit status 1


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions "3:178936057-178936106 3:178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

E0416 22:39:09.366390 140299630700288 errors.py:61] Command line parsing failure: make_examples does not accept positional arguments but some are present on the command line: "['/tmp/Bazel.runfiles_WCb7xE/runfiles/com_google_deepvariant/deepvariant/make_examples.py', '3:178952054-178952106']".

real	0m1.802s
user	0m1.471s
sys	0m0.541s
Traceback (most recent call last):
  File "/opt/deepvariant/bin/run_deepvariant.py", line 317, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/deepvariant/bin/run_deepvariant.py", line 307, in main
    subprocess.check_call(command, shell=True, executable='/bin/bash')
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106 3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}' returned non-zero exit status 1


C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions "3:178936057-178936106,178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106,178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

I0417 11:37:38.062730 140667721201408 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0417 11:37:38.096453 140667721201408 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0417 11:37:38.096697 140667721201408 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0417 11:37:38.100089 140667721201408 make_examples.py:1324] Preparing inputs
I0417 11:37:38.142340 140667721201408 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0417 11:37:38.155741 140667721201408 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
Traceback (most recent call last):
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1461, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1451, in main
    make_examples_runner(options)
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1325, in make_examples_runner
    regions = processing_regions_from_options(options)
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1250, in processing_regions_from_options
    options.exclude_calling_regions)
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 733, in build_calling_regions
    ranges.RangeSet.from_regions(regions_to_include, contig_dict))
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/third_party/nucleus/util/ranges.py", line 161, in from_regions
    return cls(ranges=from_regions(regions, contig_map=contig_map))
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/third_party/nucleus/util/ranges.py", line 113, in __init__
    for i, range_ in enumerate(ranges):
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/third_party/nucleus/util/ranges.py", line 496, in from_regions
    yield parse_literal(region, contig_map)
  File "/tmp/Bazel.runfiles_VkSeUL/runfiles/com_google_deepvariant/third_party/nucleus/util/ranges.py", line 588, in parse_literal
    'inputs that don\'t have it, or vice-versa.'.format(region_literal))
ValueError: Could not parse "3:178936057-178936106,178952054-178952106" as a region literal.  Region literals should have the form "chr:start-stop" or "chr:start" or just "chr".  A common error is to use the "chr" prefix on inputs that don't have it, or vice-versa.

real	0m1.971s
user	0m1.533s
sys	0m0.573s
Traceback (most recent call last):
  File "/opt/deepvariant/bin/run_deepvariant.py", line 317, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/deepvariant/bin/run_deepvariant.py", line 307, in main
    subprocess.check_call(command, shell=True, executable='/bin/bash')
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106,178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}' returned non-zero exit status 1

C8-C02W70QBHV2H:Lib002 weibia$ sudo docker run   -v "${INPUT_DIR}":"/input"   -v "${OUTPUT_DIR}:/output"   google/deepvariant:"${BIN_VERSION}"   /opt/deepvariant/bin/run_deepvariant   --model_type=WGS   --ref=/input/Homo_sapiens_assembly19.fasta  --reads=/input/proper.bam  --regions "3:178936057-178936106,3:178952054-178952106"  --output_vcf=/output/outputch33.vcf.gz  --output_gvcf=/output/outputchr33.g.vcf.gz  --num_shards=1 --make_examples_extra_args vsc_min_fraction_snps=0.004

***** Running the command:*****
time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106,3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}

I0417 11:38:50.004760 140059180123904 make_examples.py:377] ReadRequirements are: min_mapping_quality: 10
min_base_quality: 10
min_base_quality_mode: ENFORCED_BY_CLIENT

I0417 11:38:50.022934 140059180123904 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
W0417 11:38:50.023145 140059180123904 make_examples.py:558] No non-empty sample name found in the input reads. DeepVariant will use default as the sample name. You can also provide a sample name with the --sample_name argument.
I0417 11:38:50.026537 140059180123904 make_examples.py:1324] Preparing inputs
I0417 11:38:50.054613 140059180123904 genomics_reader.py:223] Reading /input/proper.bam with NativeSamReader
I0417 11:38:50.067181 140059180123904 make_examples.py:1248] Common contigs are [u'1', u'2', u'3', u'4', u'5', u'6', u'7', u'8', u'9', u'10', u'11', u'12', u'13', u'14', u'15', u'16', u'17', u'18', u'19', u'20', u'21', u'22', u'X', u'Y']
Traceback (most recent call last):
  File "/tmp/Bazel.runfiles_kTVJYh/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1461, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "/tmp/Bazel.runfiles_kTVJYh/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1451, in main
    make_examples_runner(options)
  File "/tmp/Bazel.runfiles_kTVJYh/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1325, in make_examples_runner
    regions = processing_regions_from_options(options)
  File "/tmp/Bazel.runfiles_kTVJYh/runfiles/com_google_deepvariant/deepvariant/make_examples.py", line 1252, in processing_regions_from_options
    raise ValueError('The regions to call is empty. Check your --regions and '
ValueError: The regions to call is empty. Check your --regions and --exclude_regions flags to make sure they are not resulting in set of empty region to process. This also happens if you use "chr20" for a BAM where contig names don't have "chr"s (or vice versa).

real	0m1.918s
user	0m1.528s
sys	0m0.542s
Traceback (most recent call last):
  File "/opt/deepvariant/bin/run_deepvariant.py", line 317, in <module>
    app.run(main)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/opt/deepvariant/bin/run_deepvariant.py", line 307, in main
    subprocess.check_call(command, shell=True, executable='/bin/bash')
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'time seq 0 0 | parallel -k --line-buffer /opt/deepvariant/bin/make_examples --mode calling --ref "/input/Homo_sapiens_assembly19.fasta" --reads "/input/proper.bam" --examples "/tmp/deepvariant_tmp_output/make_examples.tfrecord@1.gz" --gvcf "/tmp/deepvariant_tmp_output/gvcf.tfrecord@1.gz" --regions "3:178936057-178936106,3:178952054-178952106" --vsc_min_fraction_snps "0.004" --task {}' returned non-zero exit status 1