Project Limit Request: qiskit-aer - 30 GB

[mtreinish]

Project URL

https://pypi.org/project/qiskit-aer/

Does this project already exist?

• Yes

New limit

30GB

Update issue title

• I have updated the title.

Which indexes

PyPI

About the project

The Qiskit Aer project is a high performance Quantum Circuit simulator that is written in C++. It has been active since late 2018 when the first release was made. In general the size of the release is fairly large because as a complex c++ program with a number of upstream C++ dependencies we have a lot of individual wheels to support a wide combination of platforms with all our supported Python versions to ensure our users do not need to figure out how to build from source.

In the future we will be investigating using the stable ABI to reduce the wheel count (see: Qiskit/qiskit-aer#1743), but given the complexity of the C++ code and build system it might take some time before we can move to this (also as a project where runtime performance is important we'd want to ensure we don't have any performance regressions with newer Python versions by moving to the stable abi).

How large is each release?

325MB total

Currently each release is supporting 5 Python versions, in the next release that will be dropped to 4 (as this was the last release with 3.7 support).

10 windows wheels - 5 Python versions with both win32 (7MB each) and win64 (9.6MB each) - 83MB Total
9 macOS wheels - 5 Python versions of macOS x86_64 (2.3 MB each) and 4 python versions macOS arm64 (2.1MB each) - 19.9 MB total
25 Linux wheels - 5 Python versions for x86_64 (12.8 MB each), i686 (8.3MB each), s390x (7.2 MB each), ppc64le (8.2 MB each), and aarch64 (6.8 MB each) - 216 MB total

The source tarball is 6.5MB

How frequently do you make a release?

We average about 3-4 releases every 6 months. Although in general development on the project has been slowing so this frequency of release will likely decrease over time.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[hhorii]

@cmaureir I guess that you can handle this request according to the past closed issues (and sorry if my understanding is wrong).
Could you tell me what can we do to increase project limit of qiskit-aer?

[leofang]

Friendly nudge 🙂 This will be very helpful to many users.

[cmaureir]

All good @hhorii I was AFK for a couple of weeks, but I'm back.
@mtreinish have you explore using the Limited API ? https://docs.python.org/3/c-api/stable.html

It will help the project a lot, because it will allow you to have 1 wheel per architecture/OS, meaning that you don't have to have a separate wheel for each Python version, because it seems the current problem is that because of all the architecture and python versions you support, you ended up using lots of space in the project.
Implementing this, will reduce your linux wheels (for example) to 3, once per arch.

30G is quite a bit, so what do you think if we bump to 20G and you (and the team) explore the limited api to drastically reduce the amount of wheels you provide?

[leofang]

Hi @cmaureir not a Qiskit developer here, only a user, but I believe Qiskit Aer, like many other projects using PyBind11 (ex: TensorRT #2609), cannot use the limited API option as PyBind11 does not support it: pybind/pybind11#1755

[mtreinish]

@cmaureir sure 20gb is fine we really just need a bump so we can continue to release. I only asked for 30GB preemptively to prevent any potential future quota issues.

We have an issue open to track looking at the stable api (see: Qiskit/qiskit-aer#1743) but looking at the issue @leofang linked we might be blocked because of pybind11.

[cmaureir]

Hi @cmaureir not a Qiskit developer here, only a user, but I believe Qiskit Aer, like many other projects using PyBind11 (ex: TensorRT #2609), cannot use the limited API option as PyBind11 does not support it: pybind/pybind11#1755

I see, I was not aware of that limitation of pybind11. Thanks for the info.

@cmaureir sure 20gb is fine we really just need a bump so we can continue to release. I only asked for 30GB preemptively to prevent any potential future quota issues.

The project limit was increased to 20G 🎉

We have an issue open to track looking at the stable api (see: Qiskit/qiskit-aer#1743) but looking at the issue @leofang linked we might be blocked because of pybind11.

I understand the problem, no worries. Of course other tools are out that provide that, but changing the binding generator is a no-go. Let's discuss another increase once you are close to 20G, maybe some other tools will be available, or you evaluate other tools, no pressure though :p

Have a nice week!