Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blosc.set_nthreads() not working #246

Closed
ali-arbabian opened this issue Apr 6, 2021 · 6 comments
Closed

blosc.set_nthreads() not working #246

ali-arbabian opened this issue Apr 6, 2021 · 6 comments

Comments

@ali-arbabian
Copy link

ali-arbabian commented Apr 6, 2021

Hi all,
Having an issue where blosc.set_nthreads() and blosc.set_releasegil() are not actually changing the arguments inside the blosc package.
I'm checking blosc.nthreads and it stays at 8 despite the blosc.n_threads() call. I've also tried moving these lines to the top of my first entry script, still no luck.

Steps to reproduce:

import blosc   
blosc.set_nthreads(12)
blosc.set_releasegil(True)
print(blosc.print_versions())
print("nthreads: ", blosc.nthreads)

Output:

python-blosc version: 1.10.2
Blosc version: 1.21.0 ($Date:: 2020-12-22 #$)
Compressors available: ['blosclz', 'lz4', 'lz4hc', 'zlib', 'zstd']
Compressor library versions:
  BloscLZ: 2.3.0
  LZ4: 1.9.3
  Zlib: 1.2.8
  Zstd: 1.4.8
Python version: 3.7.3 (default, Mar 27 2019, 22:11:17) 
[GCC 7.3.0]
Platform: Linux-4.15.0-91-generic-x86_64 (#92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020)
Linux dist: Ubuntu 18.04.4 LTS
Processor: x86_64
Byte-ordering: little
Detected cores: 80
Number of threads to use by default: 8
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
None
nthreads:  8
@esc
Copy link
Member

esc commented Apr 6, 2021

@ali-arbabian thank you for reporting this! I have encountered this in Bloscpack too but have not gotten around to debugging it further. The following issue in Bloscpack:

Blosc/bloscpack#96

... I believe actually has it's root cause in this python-blosc bug you have reported here! I also believe this may have been broken for some time now(years?) but not sure about the exact python-blosc version where this regression was introduced . I would also consider this bug to be of reasonably high severity since it severly limits the usefulness and flexibility of the package.

@FrancescAlted
Copy link
Member

Thanks for bringing this to our attention. We will try to fix this soon. As always, if you find a solution/workaround, this would help us in accelerating the fix.

@ali-arbabian
Copy link
Author

Thanks for bringing this to our attention. We will try to fix this soon. As always, if you find a solution/workaround, this would help us in accelerating the fix.

Sounds good! I will look into this issue again this weekend if I get the chance.
Also just to clarify something in my original post, I'm not sure if blosc.set_releasegil() is working or not as there is no function to check the current releasegil state in blosc.

@ali-arbabian
Copy link
Author

Any updates on this issue?

@FrancescAlted
Copy link
Member

Ups, thanks for the reminder. The code worked fine, but internally we did not update the global nthreads variable. This should be fixed now.

@esc
Copy link
Member

esc commented Jun 1, 2021

I can confirm this works:

In [1]: import blosc

In [2]: blosc.set_nthreads(1)
Out[2]: 8

In [3]: blosc.nthreads
Out[3]: 1

In [4]: blosc.__version__
Out[4]: '1.10.3.dev0'

Previously:

In [1]: import blosc

In [2]: blosc.set_nthreads(1)
Out[2]: 8

In [3]: blosc.nthreads
Out[3]: 8

In [4]: blosc.__version__
Out[4]: '1.10.2'

This is great @FrancescAlted -- would we be able to get a new release to that I can close Blosc/bloscpack#96 -- who do I ask these days?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants