Speed up test_sorting_s3_nwb_zarr #2767
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chrishalcrow
added
testing
for more information, see https://pre-commit.ci
alejoe91
approved these changes
Apr 29, 2024
| check_sortings_equal(sorting, sorting_loaded) | ||
|
|
||
| # just take 3 random units to test | ||
| rng = np.random.default_rng(seed=2205) |
There was a problem hiding this comment.
This seed is my brithday :)
You can put your own one if you want!
There was a problem hiding this comment.
I'll celebrate your birthday every time I do a randomised test ;)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Speed up a very slow test, by only checking some units, cutting the total test run from 33->28 minutes!
A preview of the Great-Test-Speed-Up of 2024: SpikeInterface/SpikeInterface-Hackathon-Edinburgh-May24#8
(This seemed too easy and impactful to wait until the hackathon)
The function
test_sorting_s3_nwb_zarrdownloads a sorting object from s3 and checks some things about it. This has traditionally taken a while because the object is pretty big and has ~450 units. Most of the testing time was spent looping over these 450 units and calculating their spike trains. This is probably overkill, so I propose to only check a random selection of 3/450 units.You can see the overall speed-up here, from my github page (I realise these are failing, but they are attempting all the tests!!):