RuntimeError: HHSearch failed:

[mestia]

Thank you for providing the singularity image, however when running the image the way you described it here:
google-deepmind/alphafold#10 (comment)
I get this error:

I0804 10:53:22.427164 139881170892608 utils.py:36] Started HHsearch query
I0804 10:53:22.626598 139881170892608 utils.py:40] Finished HHsearch query in 0.199 seconds
Traceback (most recent call last):
  File "/app/alphafold/run_alphafold.py", line 303, in <module>
    app.run(main)
  File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/app/alphafold/run_alphafold.py", line 277, in main
    predict_structure(
  File "/app/alphafold/run_alphafold.py", line 127, in predict_structure
    feature_dict = data_pipeline.process(
  File "/app/alphafold/alphafold/data/pipeline.py", line 141, in process
    hhsearch_result = self.hhsearch_pdb70_runner.query(uniref90_msa_as_a3m)
  File "/app/alphafold/alphafold/data/tools/hhsearch.py", line 85, in query
    raise RuntimeError(
RuntimeError: HHSearch failed:
stdout:


stderr:

Any hint about what could be wrong?
Thank you!

[rsdmse]

We're not seeing this error on our platform. Have you downloaded the entire database and bind-mounted the correct paths on your machine?

Our documentation for Rivanna (UVA's HPC cluster) users can be found here. Unfortunately, it's beyond our capabilities to troubleshoot this for other institutions. Comparing with the comment you quoted, we don't have the environment variable regarding memory, but I doubt that's the issue here.

There is a similar issue reported on DeepMind's repo. I suggest that you reach out to that person to see how the issue was resolved.

[mestia]

Thank you for your quick reaction, the database is bind mounted and I'll double check the path.

[SoniaAndrys]

Hi @mestia, I am dealing with the same issue right now, have you already come up with what is wrong? Any tips? Database is mounted and my paths are correct, yet I am still getting the "HHSearch failed:" error.
I will appreciate any tip from you!

Best regards,
Sonia

[rsdmse]

I found this suggestion on the alphafold repo: google-deepmind/alphafold#97 (comment)

[SoniaAndrys]

Thank you, @rsdmse - will try :)

[seb45tian]

Hi, I ran into the same error. Issue for me was, that I build the image myself (the original deepmind one) on a skylake CPU and was trying to run it on an ivybridge node. After some diggin' I ran the /usr/bin/hhsearch command itself inside the container and got a segfault - this makes sense as it is compiled within the build procedure of the image and the container might therefore be CPU architecture specific. I hope this helps.

[rsdmse]

Our local users have not reported this issue. Please follow up with DeepMind. Thanks everyone for the input.

[yunxiangz]

Hi @rsdmse , could you explain how you increase the mem and the number of GPU, please? (e.g., with parameters in the python script? or via system settings?)
If possible, could you also share the command that you used to solve this problem?

PS: I am running locally without docker.

Thank you so much!

[cfgauss]

I can confirm that on our university HPC machines, increasing memory and cores to SLURM parameters
--mem=62GB
-c 16
got rid of this error for us.