(SV) more alignment edge cases #5044

SHuang-Broad · 2018-07-22T20:27:15Z

And another alignment edge case illustrated below (SAM records)

asm017280:tig00000	0	chr18	77371711	60	1447M50D184M980S	*	0	0	AATCAGATTTTCAGAACAGAGCTTCTTAGGGAGATGGTTGGACCACCATAGTCCTGCATCTTCAGGGAGCTGCTCACCATCCCCACTACTGCAGGCACCACTGATCCCTGAGTGAGACCCAGGCACTAACAAGACATGAAGAAGGTGCTGGGTTCCTGGCCGGTCCTTTTGACCCATAAGTGTGGTTCAGTTCTTGGCACTGATGGGCTGGTCGGATGGTGGTGAGGTCAAGCACTCCTTCAGTGCCTGAGGCGTGGCGAGTGGACTTTGGGACCTCTTCCTGAGATGATGATTGCTTCTCAAGGGTGGGAATGAGTTCTTCCAGGGGTCAGCGGTGTGTCATAGGCAAGCACAGCACATTACTAGAAAGCATACGTTTTATTAAGTAGAAGGTATCTTTAATTCCCTGAATAGTAAGACATCTTTGGATTGATGGAAAGGTTGCCCTGGGAAAGGTGGAATATTTGTTGTCATGACCTAAAGCAGCATGCCTAACACCATGTTTGCCAGAGCTCATGCCCACACTGCCATCTCAGGGTTTTCTAAGGGAGAGGAAGCAAGAGGAGGCCCTCGAATTTCTAACCGATTGCTGCAGTCAAAAAATTGCTAAGAAGGACTCGGTGGAAAAGGGAAAACTCAAAAGTGAAGTATGCTGTGCAGAAGCCAGAGCTCCACCGGCTCCCAGGATTACAGTAACAGTCAGGCTCCTGCTGTCTGGGCCCAATAACGCCATGCGCAGCATCCACATCAGCTGTCCTGAGAACCAGCAAGGCGTCCAAGGCTTAGTGTGTGTTAGAGGCCAATACTGTCCCTGCTGTTCATTCTCAGGCTAAACCCACGATCTTAGTAAGCACAGATGCAGAGACACAGTTTGTGGTATTGCCGTCATTTTCTTGGGAATGTATCAACAGAGTATACAAGGGTAACAGTTCTTTCTATAAAAAAAATCAACATTTATTTTTATTCATTAATTTACCATGAAAATGATTTTTTCCCAGTAGAGAAGTCAGTCTGGGAAAACCAGATAAAAACTAGAAATGATCTTGTTCGCTGTTGTGGAGGCGGTGCAGGAGGCCCGGGAGCTGGATCACTTGACTTTGTTTTCTCTCCTCCTCTGCTTGCAGAACATCTCTGCTACAGTGGAGGACCCACACTCATAGTTCAGCTCCCCACCTCACACCCCAGCCAAGCCCAGGACCCACACTCATGGCCCCGCTCCCCACCTCACACCCCAGCCAAGCCCAGGACCCACGCTCATGGCCCCGCTCTCCACCTCATACCCCAACCAAGTCTAGGACCCACGCTCATGGCCCTGCTCCCCACCTCACACCCCGACCAAGTCCAGGACCCACGCTCAGGACCCCTCTCCCCACCTCACACCTCAGCCACGCCCAGGACCCATGCTCACAGTTCCGCTCCCCACCTCACACCCCAACCAAGTCCAGGACCCATGCTCATGGCCCTGCTCCCCACCTCACACCCCGACCAAGTCCAGGACCCATGCTCATGGCCCCACTCTCCACCTCATACCCCAGCCAAGCCCAGGACCCACACTCACGGTTCCGCTCCCCACCTCACACCCCGACCAAGCCCAGGACCCACACTCAGGACCCCTCTCCCCACCTCACACCGCAGTTGCAGCATCTGAGGGGAGGGGCAGGACAAGGTGGCAGAGGTGGGGCATTTTTACCCAGAGAAGACCAAAAACAGTCAAAGGGATGGTTTAAATACCTGTCTTTTCTATTTTTTCACTGACTCCATGGTAACAGATTTTGCAGTAAGAACCAACATACACACATCCCTAAAACATATTTCATCAAATAATGCTGACATTATGTGTAGCACATGCTGATATTTTCTATTCTATTCTCATTTTCTCTCTCTTTTTTTCAAACAAATGATAATGAAAATCCACTAAGCTAAGGGTCATGACCAACTAACGGGTAACACCCTTCAGTTTGGAAATACTGATTTAAATTTTGGCTTGCTCTAAGCTATAATTCACATTTGACATTTAGATCATATTTTTCCCCTACTGACTCTCAAGTGGGAGCTGAGAGGCAGTACCTAATAAATACAGAGAAAAAGAAAGAAGGAAGGAAAATTGATATTTACAACACAACCAAGTTTTAATGTGAGTTAAAAAAACGGGTTTCTTTTTGATGGAAATCTTCTAAAATTGTGCTGATGGTTACACACTTCTGTGAATGCACCAAAAACCATTGAATTGTACATTTTAAATGAGGTCATTGTTGCTCTGTGAGTTATATGTCAACAAAGCCGTGAAAATCAGCAAACGAGAACAAACAGTTCTGGCCTAGGCCCTGCGAGGCCTCTTTCTAAGGCCGCCGCACGCCCATCTGGGCCATTAAACTCCTCAAGGCGCTCTGACTTCCTTGTGTAAGAGATATTGCTGTTTCTTATAAGGCACTGATGTACTTTTGCTTCCTTATTGGGGTAAAATTTAAAGACCATAACACTGATGGTTTTAAGAGTGCAGTTAAAAAAAGTTTTGACAAATGTATAAACACATGTAACAACCGCCCCAATCAATACACAGCACATTTCCACGGCCCCAGCAGGTTTCCTTGGTCCCCTTCCCAGCAAATCC	*	SA:Z:chr18,77373176,+,1165S338M50I1058M,60,93;	MD:Z:188G345G84A55T34A16T324A1G0A9C186A15C8C6G5C1C9A24T5A8T14T1G4G16C6A12C0A4T0G1C0C1T19G13^CCCACGCTCAGGACCCCTCTCCCCACCTCACACCTCAGCCACGCCCAGGG10C2T0T1C19A147	RG:Z:GATKSVContigAlignments	NM:i:88	AS:i:1375	XS:i:262
asm017280:tig00000	2048	chr18	77373176	60	1165H338M50I1058M	*	0	0	CTCCCCACCTCACACCCCAGCCAAGCCCAGGACCCACACTCATGGCCCCGCTCCCCACCTCACACCCCAGCCAAGCCCAGGACCCACGCTCATGGCCCCGCTCTCCACCTCATACCCCAACCAAGTCTAGGACCCACGCTCATGGCCCTGCTCCCCACCTCACACCCCGACCAAGTCCAGGACCCACGCTCAGGACCCCTCTCCCCACCTCACACCTCAGCCACGCCCAGGACCCATGCTCACAGTTCCGCTCCCCACCTCACACCCCAACCAAGTCCAGGACCCATGCTCATGGCCCTGCTCCCCACCTCACACCCCGACCAAGTCCAGGACCCATGCTCATGGCCCCACTCTCCACCTCATACCCCAGCCAAGCCCAGGACCCACACTCACGGTTCCGCTCCCCACCTCACACCCCGACCAAGCCCAGGACCCACACTCAGGACCCCTCTCCCCACCTCACACCGCAGTTGCAGCATCTGAGGGGAGGGGCAGGACAAGGTGGCAGAGGTGGGGCATTTTTACCCAGAGAAGACCAAAAACAGTCAAAGGGATGGTTTAAATACCTGTCTTTTCTATTTTTTCACTGACTCCATGGTAACAGATTTTGCAGTAAGAACCAACATACACACATCCCTAAAACATATTTCATCAAATAATGCTGACATTATGTGTAGCACATGCTGATATTTTCTATTCTATTCTCATTTTCTCTCTCTTTTTTTCAAACAAATGATAATGAAAATCCACTAAGCTAAGGGTCATGACCAACTAACGGGTAACACCCTTCAGTTTGGAAATACTGATTTAAATTTTGGCTTGCTCTAAGCTATAATTCACATTTGACATTTAGATCATATTTTTCCCCTACTGACTCTCAAGTGGGAGCTGAGAGGCAGTACCTAATAAATACAGAGAAAAAGAAAGAAGGAAGGAAAATTGATATTTACAACACAACCAAGTTTTAATGTGAGTTAAAAAAACGGGTTTCTTTTTGATGGAAATCTTCTAAAATTGTGCTGATGGTTACACACTTCTGTGAATGCACCAAAAACCATTGAATTGTACATTTTAAATGAGGTCATTGTTGCTCTGTGAGTTATATGTCAACAAAGCCGTGAAAATCAGCAAACGAGAACAAACAGTTCTGGCCTAGGCCCTGCGAGGCCTCTTTCTAAGGCCGCCGCACGCCCATCTGGGCCATTAAACTCCTCAAGGCGCTCTGACTTCCTTGTGTAAGAGATATTGCTGTTTCTTATAAGGCACTGATGTACTTTTGCTTCCTTATTGGGGTAAAATTTAAAGACCATAACACTGATGGTTTTAAGAGTGCAGTTAAAAAAAGTTTTGACAAATGTATAAACACATGTAACAACCGCCCCAATCAATACACAGCACATTTCCACGGCCCCAGCAGGTTTCCTTGGTCCCCTTCCCAGCAAATCC	*	SA:Z:chr18,77371711,+,1447M50D184M980S,60,88;	MD:Z:16T6C7G4T0G4C2T0T22A5T10T12A19G5C1C9A4C2T0T1C26C11A28C6A12C0A5G18A5G6C10C5C6A70G4T10T0G4T1G4G3T200G738G52	RG:Z:GATKSVContigAlignments	NM:i:93	AS:i:1115	XS:i:0

codecov-io · 2018-07-22T21:23:37Z

Codecov Report

Merging #5044 into master will decrease coverage by 6.011%.
The diff coverage is 88.496%.

@@               Coverage Diff               @@
##              master     #5044       +/-   ##
===============================================
- Coverage     86.382%   80.371%   -6.011%     
+ Complexity     28674     27677      -997     
===============================================
  Files           1785      1785               
  Lines         132713    133530      +817     
  Branches       14766     15004      +238     
===============================================
- Hits          114640    107319     -7321     
- Misses         12746     21030     +8284     
+ Partials        5327      5181      -146

Impacted Files	Coverage Δ	Complexity Δ
...e/ContigChimericAlignmentIterativeInterpreter.java	`80.645% <0%> (-1.772%)`	`31 <0> (ø)`
...te/hellbender/tools/spark/sv/utils/SVInterval.java	`82.979% <0%> (-1.804%)`	`32 <3> (+3)`
...ender/tools/spark/sv/utils/SVIntervalUnitTest.java	`95.652% <100%> (+12.319%)`	`4 <2> (+2)`	⬆️
...lignment/AssemblyContigAlignmentsConfigPicker.java	`91.563% <71.429%> (-1.953%)`	`108 <13> (+13)`
.../AssemblyContigAlignmentsConfigPickerUnitTest.java	`99.057% <96.923%> (-0.548%)`	`30 <10> (+10)`
...read/markduplicates/MarkDuplicatesSparkTester.java	`0% <0%> (-100%)`	`0% <0%> (-5%)`
...tils/read/markduplicates/MarkDuplicatesTester.java	`0% <0%> (-100%)`	`0% <0%> (-2%)`
...rs/variantutils/SelectVariantsIntegrationTest.java	`0.275% <0%> (-99.725%)`	`1% <0%> (-65%)`
.../AbstractMarkDuplicatesCommandLineProgramTest.java	`0.443% <0%> (-99.335%)`	`2% <0%> (-74%)`
...kers/filters/VariantFiltrationIntegrationTest.java	`0.826% <0%> (-99.174%)`	`1% <0%> (-25%)`
... and 140 more

cwhelan

I have a couple of refactoring suggestions and one implementation question but otherwise this looks ok. Make sure you run it on the sample that was failing to test it out.

cwhelan · 2018-07-23T01:44:22Z

...tute/hellbender/tools/spark/sv/discovery/alignment/AssemblyContigAlignmentsConfigPicker.java

@@ -571,13 +572,30 @@ private static double computeTigExplainQualOfOneConfiguration(final List<Alignme
            return bestConfigurations.stream()
                    .map(mappings -> splitGaps(mappings, false))
                    .map(mappings -> removeNonUniqueMappings(mappings, ALIGNMENT_MQ_THRESHOLD, ALIGNMENT_LOW_READ_UNIQUENESS_THRESHOLD))
+                    .filter(mappings ->  {


Would this be cleaner like this?

.filter(mappings -> mappings.goodMappings.size != 2 || ! simpleChimeraWithStichableAlignments(mappings.goodMappings.get(0), mappings.goodMappings.get(1))))

cwhelan · 2018-07-23T01:45:11Z

...tute/hellbender/tools/spark/sv/discovery/alignment/AssemblyContigAlignmentsConfigPicker.java

                    .map(mappings -> createContigGivenClassifiedAlignments(contigName, contigSeq, mappings, true))
                    .sorted(getConfigurationComparator())
                    .iterator();
        } else {
            final GoodAndBadMappings intermediate = splitGaps(bestConfigurations.get(0), false);
            final GoodAndBadMappings result = removeNonUniqueMappings(intermediate, ALIGNMENT_MQ_THRESHOLD, ALIGNMENT_LOW_READ_UNIQUENESS_THRESHOLD);
-            return Collections.singletonList(createContigGivenClassifiedAlignments(contigName, contigSeq, result, false)).iterator();
+            if (result.goodMappings.size() != 2


If you do the refactoring suggested above you could make a little method for the if clause and only write it once.

cwhelan · 2018-07-23T01:51:49Z

...tute/hellbender/tools/spark/sv/discovery/alignment/AssemblyContigAlignmentsConfigPicker.java

@@ -853,11 +863,14 @@ private static void removeDueToShortReadSpan(final List<AlignmentInterval> selec
        }
    }

+    @VisibleForTesting
+    static final int TEMPORARY_SELF_POINTING_SV_INTERVAL_CONTIG = 0;


I don't understand this. It doesn't seem like you ever refer to that except when putting things into the max overlap map below. Why create these invalid intervals? Also, I think that our contig numbering scheme for SVInterval starts at zero so these fake intervals will look like real intervals. I think I'd much rather keep them the old way in Tuple2<Integer, Integer>s than jam them into invalid interval objects.

cwhelan · 2018-07-23T01:52:18Z

...tute/hellbender/tools/spark/sv/discovery/alignment/AssemblyContigAlignmentsConfigPicker.java

+     * See the funny alignment signature described in ticket 4951 on GATK github
+     * @param intervalOne assumed to start no later   than {@code intervalTwo} on the read
+     * @param intervalTwo assumed to start no earlier than {@code intervalOne} on the read
+     * @return if the two given intervals can be stitched together


SHuang-Broad · 2018-07-23T15:57:37Z

All 4 trios pass without problems, by taking in the assembly bams from Ted's branch.

cwhelan

Looks good, thanks for making those changes.

👍

* ticket 4951 * two neighboring gapped alignments, after gap split, generate unsorted alignments * also a new utility method in SVInterval

SHuang-Broad · 2018-07-23T16:42:55Z

Thanks for the quick review!

SHuang-Broad added the SV label Jul 22, 2018

SHuang-Broad requested a review from cwhelan July 22, 2018 20:27

SHuang-Broad added the PRIORITY_HIGH label Jul 22, 2018

cwhelan requested changes Jul 23, 2018

View reviewed changes

cwhelan approved these changes Jul 23, 2018

View reviewed changes

(SV) addressing more edge cases in assembly alignments

a4b63ce

* ticket 4951 * two neighboring gapped alignments, after gap split, generate unsorted alignments * also a new utility method in SVInterval

SHuang-Broad force-pushed the sh-sv-more-alignment-edge-cases branch from 1266952 to a4b63ce Compare July 23, 2018 16:42

SHuang-Broad merged commit 9d9484f into master Jul 23, 2018

SHuang-Broad deleted the sh-sv-more-alignment-edge-cases branch July 23, 2018 18:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(SV) more alignment edge cases #5044

(SV) more alignment edge cases #5044

SHuang-Broad commented Jul 22, 2018

codecov-io commented Jul 22, 2018 •

edited

Loading

cwhelan left a comment

cwhelan Jul 23, 2018

SHuang-Broad Jul 23, 2018

cwhelan Jul 23, 2018

SHuang-Broad Jul 23, 2018

cwhelan Jul 23, 2018

cwhelan Jul 23, 2018

SHuang-Broad Jul 23, 2018

SHuang-Broad commented Jul 23, 2018

cwhelan left a comment

SHuang-Broad commented Jul 23, 2018

(SV) more alignment edge cases #5044

(SV) more alignment edge cases #5044

Conversation

SHuang-Broad commented Jul 22, 2018

codecov-io commented Jul 22, 2018 • edited Loading

Codecov Report

cwhelan left a comment

Choose a reason for hiding this comment

cwhelan Jul 23, 2018

Choose a reason for hiding this comment

SHuang-Broad Jul 23, 2018

Choose a reason for hiding this comment

cwhelan Jul 23, 2018

Choose a reason for hiding this comment

SHuang-Broad Jul 23, 2018

Choose a reason for hiding this comment

cwhelan Jul 23, 2018

Choose a reason for hiding this comment

cwhelan Jul 23, 2018

Choose a reason for hiding this comment

SHuang-Broad Jul 23, 2018

Choose a reason for hiding this comment

SHuang-Broad commented Jul 23, 2018

cwhelan left a comment

Choose a reason for hiding this comment

SHuang-Broad commented Jul 23, 2018

codecov-io commented Jul 22, 2018 •

edited

Loading