-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
can't register atexit after shutdown thrown in Python 3.9 #3221
Comments
Hi @relativityboy, thanks for bringing this to our attention. This appears to be an issue specific to the S3 resource. I wasn't able to reproduce it on other resources (ec2, dynamodb, etc). There was a change in Python 3.9 to prevent daemon threads in concurrent.futures due to unexpected behaviors with subprocesses. We'll need more investigation but from first glance this is likely an issue with the transfer module or |
I have another example of this that is occurring with a highly upvoted SO answer for uploading logs to S3 using atexit.register to register the uploader function during python shutdown. I suspect this registered hook is being cloned into some sort of multiprocessing under the hood and it's attempting to call the hook for every subprocess (thread?). Here is the SO answer that appears to now be a broken pattern: https://stackoverflow.com/a/51070892/2970906 Code sample below to reproduce:
Python 3.9.9 |
Any update on this issue? |
To address this issue we recommend joining your child threads before letting the main thread go into shutdown. But as mentioned in an earlier comment some investigation is still required to look into how this might be handled in s3transfer. |
@tim-finnigan I'm not quite understanding how to implement a work around in practice. For instance, if you take the original example:
If your goal is to just return an S3 client/session, I don't think you would want to add:
... If I understand it, this would call the function again during tear down. Any help would be much appreciated. |
Hi @marcrleonard, For this specific example, you have to join your threads. t = Timer(0.5, x)
t.start()
t.join() The problem with the current implementation is you're starting a ThreadPoolExecutor inside of a Thread without notifying the parent thread. This is resulting in timer ending before cleanup is performed on S3transfer. The core of the issue stems from changes in 3.9 around concurrent.futures works with ThreadPoolExecutor. You can find the CPython issue here. There isn't straight forward way to fix this with Python's new behavior at the moment unfortunately. Edit: To make sure I conveyed the scope of the issue, this isn't specific to any of our code. It's also reproducible with just concurrent.futures code. It's a fundamental issue with Python's threading paradigm. from threading import Timer
def x():
from concurrent.futures import ThreadPoolExecutor
Timer(0.5, x).start() |
Does anyone have a workaround for this case? I'm currently stuck with python 3.9 EDIT: neveremind, just found this other thread python/cpython#86813 (comment) |
I managed to get this sorted by importing the ThreadPoolExecutor module, before calling any of my main application code - so at the top of my
Just the act of importing the module early (before any threads are initialised) was enough to fix the error. There is an on-going issue around how this module must be present in the main thread, before any child threads import or use any related threading library code. The inspiration for this fix came from this post on the Python bugs site. My issue was specifically around |
(Reference: python/cpython#86813 (comment)) You can make this work by disabling threading in S3 client. import boto3.s3.transfer
client = boto3.client("s3")
client.upload_file(
Filename="path/to/file",
Bucket="my-bucket",
Key="my-file",
Config=boto3.s3.transfer.TransferConfig(use_threads=False),
) |
Describe the bug
When attempting to create an s3 session resource running on python 3.9 function call throws with message
can't register atexit after shutdown
Behaves as expected in python
3.8
Expected Behavior
When running in python
3.9
the following code (assuming valid values) results in s3 being a resourceCurrent Behavior
When running in python
3.9
the following code (assuming valid values) results in an ExceptionReproduction Steps
Run the following in python 3.9 with
public_key
,secret_key
, andaws_region
set to valid valuesPossible Solution
Run with python 3.8? Error does not occur.
Additional Information/Context
No response
SDK version used
1.21.38
Environment details (OS name and version, etc.)
Mac OS 12.3.1, Python 3.9
The text was updated successfully, but these errors were encountered: