Speed up combine_wave_lists using new merge_sorted function #1243
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@jchiang87 found that a significant amount of time in some recent imSim runs was spend doing the
combine_wave_list
function when adding the bulge+disk+knots galaxy components. This is partly because the SEDs probably have more wavelengths than we really need. But regardless, this function was not the most efficient bit of code. This PR significantly improves the algorithm. In the typical case, it's more than 3x faster. And when all the inputs are the same (which was the case for imSim in Jim's test) it's almost 5x faster.The new underlying functionality is a new utility function
merge_sorted
, which merges two or more numpy arrays when the inputs are all already sorted. This is a pretty simple function (familiar to those who know merge sort), but there is apparently no native numpy function that does this. I did find sortednp who also implements this, but I just went ahead and wrote my own rather than add a new dependency.The timing script I wrote compares the new version with the old one, and with Jim's suggested change of checking for equality in order to skip the merge entirely. (The new version also includes this check, btw.) Here are the timing results on my laptop: