-
-
Notifications
You must be signed in to change notification settings - Fork 30.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-116738: Make _csv module thread-safe #118344
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, Sorry for the late.
@colesbury Do you have any comments about this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the long delay in reviewing this. Two requests:
- Let's change
field_limit
to aPy_ssize_t
. That seems more appropriate here and avoids the need for adding new atomic bindings. I'd prefer to avoid adding additional atomic bindings because its more code to maintain and its easy to add subtle bugs in the atomic bindings. Additionally,field_limit
is compared tofield_len
, which is already aPy_ssize_t
. - Let's use relaxed atomics for reading and writing
field_limit
, e.g.,FT_ATOMIC_STORE_SSIZE_RELAXED
. Relaxed ordering is fine for these sorts of global settings.
For the data type change:
%ld
->%zd
PyLong_AsLong
->PyLong_AsSsize_t
Thanks for the review, updated! |
Thanks @aisk for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
(cherry picked from commit a00221e) Co-authored-by: AN Long <aisk@users.noreply.github.com>
GH-125328 is a backport of this pull request to the 3.13 branch. |
This extension module has some module state values which may be read or written in different threads:
field_limit
: A simple integer value to store the maximum field size for the parser;dialects
: A dict value to store all the dialects by string names.field_limit
There is a public function
field_size_limit
that can read and write it,using a critical section to protect it. For other internal usages,_Py_atomic_load_long
and_Py_atomic_store_long
are used to make it thread safe.dialects
There are three public functions
get_dialect
,unregister_dialect
andlist_dialects
that can access this field.For the first two functions, a critical section is used to protect it.And the
list_dialects
function just returns the dict's keys withPyDict_Keys
. I think its result is readonly andPyDict_Keys
itself is thread safe, so we don't need to do anything.Other internal usages of
dialects
involve accessing its value withget_dialect_from_registry
, which simply callsPyDict_GetItemRef
and returns the result. I think we don't need to change it too.Update: I think these APIs is just call
dict
APIs which is already thread-safe, and there are no other state changes inside these APIs, and no intermediate variables need to protect in these functions, so we don't need add locks on them.