cannot allocate memory in static TLS block exception raised when pyarrow is imported before MySQLdb

[mdobrzanski]

Describe the bug

The mysqlclient (MySQLdb) cannot be imported if pyarrow is imported first. An ImportError is raised: /lib64/libstdc++.so.6: cannot allocate memory in static TLS block

[root@930fd58ceb8a /]# python3 -c 'import pyarrow; import MySQLdb; print("OK")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib64/python3.9/site-packages/MySQLdb/__init__.py", line 17, in <module>
    from . import _mysql
ImportError: /lib64/libstdc++.so.6: cannot allocate memory in static TLS block

I have originally created the issue with reproduction steps at pyarrow project apache/arrow#39854 . The maintainer of python pyarrow thinks that the issue has to be fixed on the MySQLdb side. Do you think you can work on that?

Environment

I've confirmed it is a problem on Oracle Linux 9 and Ubuntu 22.04

How did you install libmysqlclient libraries?

with dnf using mysql community repository v.

What version of mysqlclient do you use?

2.2.1 or 2.2.4

[methane]

Note that this library uses/wraps/links libmysqlclient. We don't provide libmysqlclient. So I can not fix it.

I've confirmed it is a problem on Oracle Linux 9 and Ubuntu 22.04

Oh, really? I thought Ubuntu had fixed this issue already.
https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1889851

[methane]

with dnf using mysql community repository v.

Have you installed libmysqlclient with dnf even on Ubuntu? Or do you used apt?

Ubuntu package contains patch to disable STATIC_TLS.
But if you installed MySQL from MySQL community edition repository, the patch is not applied.

I can not reproduce your issue when I tried on Ubuntu 22.04 with libmysqlclient-dev from Ubuntu repository.

methane@skyland:~/pymysql-702$ mkvenv
(pymysql-702) methane@skyland:~/pymysql-702$ sudo apt search libmysqlclient
...
libmysqlclient-dev/mantic-updates,mantic-security 8.0.36-0ubuntu0.23.10.1 amd64
...
(pymysql-702) methane@skyland:~/pymysql-702$ sudo apt install libmysqlclient-dev
...
(pymysql-702) methane@skyland:~/pymysql-702$ pip install mysqlclient pyarrow
...
Collecting mysqlclient
  Downloading mysqlclient-2.2.4.tar.gz (90 kB)
...
Collecting pyarrow
  Obtaining dependency information for pyarrow from https://files.pythonhosted.org/packages/85/55/636f006d963ddf77270fd294163e149b0719
aaaf794de0d023aee88f6335/pyarrow-15.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata
  Downloading pyarrow-15.0.0-cp311-cp311-manylinux_2_28_x86_64.whl.metadata (3.0 kB)
Collecting numpy<2,>=1.16.6 (from pyarrow)
  Obtaining dependency information for numpy<2,>=1.16.6 from https://files.pythonhosted.org/packages/3a/d0/edc009c27b406c4f9cbc79274d6
e46d634d139075492ad055e3d68445925/numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB 7.8 MB/s eta 0:00:00
Downloading pyarrow-15.0.0-cp311-cp311-manylinux_2_28_x86_64.whl (38.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.3/38.3 MB 62.9 MB/s eta 0:00:00
Downloading numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.3 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 78.7 MB/s eta 0:00:00
...
(pymysql-702) methane@skyland:~/pymysql-702$ python -c 'import MySQLdb; import pyarrow; print("ok")'
ok
(pymysql-702) methane@skyland:~/pymysql-702$ python -c 'import pyarrow; import MySQLdb; print("ok")'
ok

[wimglenn]

I also hit this issue, on RHEL8.

$ python3 -c 'import MySQLdb, pyarrow'
$ python3 -c 'import pyarrow, MySQLdb'
Traceback (most recent call last):
  File "..../lib/python3.11/site-packages/MySQLdb/__init__.py", line 18, in <module>
    from . import _mysql
ImportError: /lib64/libstdc++.so.6: cannot allocate memory in static TLS block

I suspect it's because they switched from global-dynamic to initial-exec static tls.

$ readelf -ld /usr/lib64/mysql/libmysqlclient.so.21 | grep TLS
  TLS            0x00000000004482a0 0x00000000006482a0 0x00000000006482a0
 0x000000000000001e (FLAGS)              BIND_NOW STATIC_TLS

I was able to dodge the issue by linking mysqlclient against libmariadb.so instead of libmysqlclient.so.