Replace File.createTempFile with IOUtils.createTempFile

[lbergelson]

• This ensure that temp files are deleted on shutdown
• Fixes Gatk leaving behind config files #6771

[lbergelson]

@jamesemery mind taking a look at this?

[jamesemery]

@lbergelson can you explain to me why this fixes the issue with tmp files?

[jamesemery]

Ahh... I see. Our IOUtils.createTmpFile() doesn't actually do the same thing as File.createTempFile() it secretly creates a directory to house everything which happens to usually capture our indexes and secondary files. I almost think we should rename IOUtils.createTmpFile() to IOUtils.createTmpFileHousedInTmpDir() so its a little clearer. Also perhaps we should find a solution to scare people away from using the Files version in the future?

[lbergelson]

@jamesemery The big difference is that IOUtils marks the files for deletion on shutdown while Files doesn't.

[lbergelson]

@jamesemery Also, since createTmpFile gives absolutely 0 information about where the temp file will be located it is completely fine to place each one in it's own unique directory, or on it's own moon or something.

[jamesemery]

God that method is so much worse than i thought... We resort to hard coding common side outputs that are likely to show up and mark them for deletion in case we happen to produce one of them... I wonder if it is indeed possible to flag a directory for deletion and if we cant improve that method by secretly putting the file you ask for into a delete-on-exit folder so we don't have to guess about indexes and side outputs (that frankly don't look complete? What if my test has a tmp-fast dict/index those aren't included in the list of side outputs to flag for deletion).

[lbergelson]

Deleting the whole folder is a good idea and we should do that instead of the targeted index indexes.

[jamesemery]

@lbergelson do you want to try implementing the folder deletion in this PR or should we spin that approach off separately?

[lbergelson]

@jamesemery No, lets do that as a separate pr.

[jamesemery]

Should we worry about this no longer catching the exception? Now this will err on this line throw new GATKException("Cannot create temp file: " + ex.getMessage(), ex); which seems less specific but probably equally as informative? This is probably fine?

[jamesemery]

This all looks fine, unless I missed something these were all trivial swaps mostly in the test files so I would say go ahead and merge this. I will go ahead and make a ticket to track the potential improvement to this code we discussed.

[lbergelson]

rebased an rerunning tests.

[codecov]

Codecov Report

Merging #6780 (de57310) into master (7f40444) will increase coverage by 0.004%.
The diff coverage is 97.590%.

Additional details and impacted files

@@               Coverage Diff               @@
##              master     #6780       +/-   ##
===============================================
+ Coverage     86.643%   86.647%   +0.004%     
- Complexity     38964     38970        +6     
===============================================
  Files           2337      2337               
  Lines         182747    182763       +16     
  Branches       20067     20067               
===============================================
+ Hits          158338    158359       +21     
+ Misses         17366     17365        -1     
+ Partials        7043      7039        -4     

Impacted Files	Coverage Δ
...r/tools/walkers/varianteval/AlleleFrequencyQC.java	90.909% <0.000%> (+5.195%)	⬆️
...e/spark/datasources/VariantsSparkSinkUnitTest.java	85.075% <0.000%> (+0.630%)	⬆️
...er/utils/MergeAnnotatedRegionsIntegrationTest.java	93.333% <ø> (ø)
.../hellbender/tools/genomicsdb/GenomicsDBImport.java	83.594% <100.000%> (-0.043%)	⬇️
...spark/sv/utils/SingleSequenceReferenceAligner.java	80.282% <100.000%> (-0.274%)	⬇️
...g/broadinstitute/hellbender/utils/io/Resource.java	62.963% <100.000%> (ø)
...tils/CombineSegmentBreakpointsIntegrationTest.java	93.671% <100.000%> (ø)
...geAnnotatedRegionsByAnnotationIntegrationTest.java	100.000% <100.000%> (ø)
...number/utils/TagGermlineEventsIntegrationTest.java	100.000% <100.000%> (ø)
...nterval/SimpleAnnotatedIntervalWriterUnitTest.java	100.000% <100.000%> (ø)
... and 29 more