Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support filling an SChunk with a specific value. This fixes #250 #254

Merged
merged 2 commits into from
Sep 12, 2024

Conversation

martaiborra
Copy link
Collaborator

No description provided.

Copy link
Member

@FrancescAlted FrancescAlted left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! I just have a couple a suggestions. If possible, please also make sure that the different NDArray constructors use this new way of building SChunks.

Also, it would be nice if you can attach a message in the PR with the improvements in speed for e.g. blosc2.zeros() and blosc2.full()

src/blosc2/blosc2_ext.pyx Outdated Show resolved Hide resolved
tests/test_schunk_fill.py Outdated Show resolved Hide resolved
@martaiborra
Copy link
Collaborator Author

Ran a bench comparing 3 ways of filling a SChunk. The output is the following:

Time for filling the schunk with `data` argument in the constructor: 0.011s
Time for filling the schunk without passing directly the value: 0.009s
Time for filling the schunk passing directly the value to `fill_special`: 0.006s

@martaiborra martaiborra merged commit f5a9afb into main Sep 12, 2024
9 checks passed
@martaiborra martaiborra deleted the special branch September 12, 2024 09:31
@FrancescAlted
Copy link
Member

Great work!

FWIW, I simplified the benchmark (essentially letting the chunksize to be computed automatically) and pushed the size of the schunk (or better, the ancillary NumPy array) towards the limits of my machine (Intel i13900K, 64 GB RAM), and here are the results:

(python-blosc2) faltet@beast:~/blosc/python-blosc2$ python bench/fill_special.py
Filling a SChunk with 2.0 Gelements of dtype=dtype('float64')
Time with `data` argument in constructor:            1553.859 ms
Time without passing directly the value:             1580.115 ms
Time passing directly the value to `fill_special`:      0.072 ms
(python-blosc2) faltet@beast:~/blosc/python-blosc2$ python bench/fill_special.py pers
Filling a SChunk with 2.0 Gelements of dtype=dtype('float64')
Writing output to bench_fill_special.b2frame...
Time with `data` argument in constructor:            1556.498 ms
Time without passing directly the value:             1653.402 ms
Time passing directly the value to `fill_special`:     47.593 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants