Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DataLoader worker (pid 1 4698) is killed by signal: Killed Error #265

Open
cauliflower0077 opened this issue Aug 19, 2024 · 0 comments
Open

Comments

@cauliflower0077
Copy link

I tried to train DINO on an original combined dataset consisting of about 2000 training images and about 500 validation images.
However, I got the following error. Please tell me how to resolve it.

File "/venv/lib/python3.10/site-packages/torch/_compile.py", line 24, in inner
    return torch._dynamo.disable(fn, recursive)(*args, **kwargs)
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/decorators.py", line 50, in disable
    return DisableContext()(fn)
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 410, in __call__
    (filename is None or trace_rules.check(fn))
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 3378, in check
    return check_verbose(obj, is_inlined_call).skipped
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 3361, in check_verbose
    rule = torch._dynamo.trace_rules.lookup_inner(
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 3457, in lookup_inner
    if check_file(filename, is_direct_call).skipped:
  File "/venv/lib/python3.10/site-packages/torch/_dynamo/trace_rules.py", line 3279, in check_file
    if any(filename.startswith(d) for d in get_legacy_mod_inlinelist()):
  File "/venv/lib/python3.10/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 14698) is killed by signal: Killed.

Firstly, I had success using 500 training images and 100 validation images, but now I would like to train with a larger number of images.
Additionally, I attempted to fix PyTorch DataLoader. Could you please let me know where I can find the 'pin_memory=true' option in DINO repository?
I want to create a robust AI model, so ‘number of class‘ is set to 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant