"a bites-like object is required, not 'str'" error #75
Comments
|
Hi Jordi, Yes that looks like a bug, it is unclear if this is related to python 3.6 due to the rather cryptic error message. Could you rerun the offending run with the debug command-line flag set: longbow --debug --hosts mypath/hosts.conf mdrun_mpi -deffnm example That should generate the full python traceback for the error, then could you paste it back here and I will then try and reproduce the error to fix it. Thanks for reporting this. |
|
Hi James, From the traceback it seems that the issue is related with module not being installed in the computer I am using to run the tests. Here is the full traceback During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
That looks like the bash command module is missing. Is this a machine that does not use linux modules? So if you ssh'd in and ran module avail, does this work? Either way, there is still a bug here because it should have told you that there is an error with the modules and not that weird error about byte-like objects. |
|
Yes, this is a machine that does not use linux modules. |
|
I've seen this before on a number of smaller cluster type machines that are managed by the research group themselves. So what is happening is Longbow is trying to load up the linux module if the executable is not an absolute path (this poses problems when users add it to their bashrc, I have yet to find a robust workaround for that!). If you specify the mdrun_mpi as an absolute path like this: longbow --hosts mypath/hosts.conf /absolute/path/to/mdrun_mpi -deffnm example Then it should disable the module load, and hopefully this should resolve the underlying issue with the modules, although I will still fix that error message so that it gives the proper one to the user. |
|
The call to module is still triggered. Note that the call to gromacs executable is 5.X style andmaybe that is an issue? /absolute/path/to/gmx_mpi mdrun -deffnm example |
|
Actually, I just realised. This is related to a different part of the code, I misread the traceback. This is relating to the environment fix I implemented for those other small cluster machines I mentioned. So on those machines, they were still using linux modules but the environment in /etc/profile was not loaded when using a non-login shell (which python subprocess doesn't). So I have a check that does this "module avail" and then sources the /etc/profile if this fails. This fixed a problem that was found at 5/6 different universities. In your case, your machine is simply not using the linux module system, it also looks like the output of that check is somehow not decoded from bytes. This is definitely a bug in Longbow, I've just complete your slurm-gres request so will look at this now. |
|
Hi Jordi, So I think I have found the offending code that caused the weird error message in the first place. Are you comfortable with testing out development copies of Longbow? If you go here https://github.com/HECBioSim/Longbow/tree/release-prep this is the release-prep branch for v1.5.1. If you download the zip file by clicking the "clone or download" button on the right, then unpack the zip. Then uninstall the old Longbow using "pip uninstall longbow" then change into the newly downloaded directory and do "pip install . --user" or without the --user if that is how you normally do it. Doing longbow --version should show v1.5.1-dev This version should contain this fix, and also the slurm-gres from the other ticket (instructions on how to use it are in that ticket, I'll need to document it still). Hopefully this should let you get past the issue you are having here, I'm not sure yet if you can get away with doing: longbow --hosts mypath/hosts.conf mdrun_mpi -deffnm example you might still need to do the absolute path to the executable: longbow --hosts mypath/hosts.conf /absolute/path/to/mdrun_mpi -deffnm example If this works, we could look at trying to come up with a smarter solution for disabling modules than having to specify the path, although this is only generally an issue on a small number of machines. |
|
Ok, I am little bit confused. I donwloaded the development version and run longbow and the error has disapeared, however, it does not look as it is actually working. This is the tracebak I get using this command. longbow --verbose --hosts /mypath/hosts.conf /mypath/mdrun_mpi -s example.tpr -deffnm output 2017-11-07 10:42:08 - This software was developed as part of the EPSRC-funded HECBioSim project (http://www.hecbiosim.ac.uk/) But nothing appears in the remote host queue nor any files are transferred. What am I doing wrong? |
|
Hi, Can you change the --verbose for --debug please. There is still a problem here, a full debug traceback should give me some more information to see what is failing. |
You are not doing anything wrong, I think you have just found some bugs in our code which we don't see getting triggered on most HPC machines. I'm grateful that you have reported them so that these can be fixed and new unit tests written to help prevent them re-occurring. |
|
Hi James, here is the debug traceback 2017-11-07 12:06:32 - INFO - longbow - Welcome to Longbow! I have double checked that /mypath/mdrun_mpi exist in the remote host. |
|
Thank you, the traceback confirms this is a bug, it is a different bug to the one previously fixed but related to the same issue. I'll issue a fix for this shortly. |
|
Hi Jordi, So the problem was related to how longbow analyses the commandline, what it does is try to identify required files and other files that need uploading based on which MD program is being used. The use of absolute paths on the executable created problems with this (a python keyerror), I have applied further patches to the 1.5.1-dev version now to fix this error. Hopefully longbow should now work normally for you. Could you verify that the latest 1.5.1-dev now works for your case please? By uninstalling and reinstalling 1.5.1-dev as you did previously. |
|
Hi James, I would say that this issue is now fixed as running longbow in debug mode did not render any weird errors. FYI: I am still unable to run the examples, but I don't think is longbow related at the moment, rather PLUMED screwing up my gromacs libraries ^_^'. In any case I am planing to intensively use longbow in the next few weeks, so expect a little (or a lot) more annoyance on this end. Thank you very much! Jordi |
|
Hi Jordi,
Excellent, I will close this ticket. If you do notice anything else related to this then let me know.
PLUMED does that from time to time. There might also be some issues with some of the input files, I think the gromacs ones are incompatible with the 2016.3+ lines of gromacs. Possibly due to removal of something they had deprecated, I have had a similar problem in our benchmark suite. I will update the examples shortly.
Power users are always welcome, just file a ticket if you see anything wrong/unexpected or need anything new, I'm always happy to help. |
Hi James,
I am having problems to run the quick-start examples for amber and gromacs (have not tested the rest). I keep getting this rather cryptic error: a bytes-like object is required, not 'str'
I am pasting the whole log file after executing this command in mypath/longbow-examples/quick_start/gromacs:
longbow --hosts mypath/hosts.conf mdrun_mpi -deffnm example
2017-10-31 17:52:56 - Welcome to Longbow!
2017-10-31 17:52:56 - This software was developed as part of the EPSRC-funded HECBioSim project (http://www.hecbiosim.ac.uk/)
2017-10-31 17:52:56 - HECBioSim facilitates high-end biomolecular simulation on resources such as ARCHER
2017-10-31 17:52:56 - Longbow is Copyright (C) of Science and Technology Facilities Council and The University of Nottingham.
2017-10-31 17:52:56 - Longbow was created by Dr James T. Gebbie-Rayet, Dr Gareth B. Shannon and Prof Charles A. Laughton.
2017-10-31 17:52:56 - Please cite our paper: Gebbie-Rayet, J, Shannon, G, Loeffler, H H and Laughton, C A 2016 Longbow: A Lightweight Remote Job Submission Tool. Journal of Open Research Software, 4: e1, DOI: http://dx.doi.org/10.5334/jors.95
2017-10-31 17:52:56 - Python version: 3.6
2017-10-31 17:52:56 - Longbow version: 1.5.0
2017-10-31 17:52:56 - Longbow Commandline: --hosts mypath/hosts.conf mdrun_mpi -deffnm example
2017-10-31 17:52:56 - hosts file is: 'mypath/hosts.conf'
2017-10-31 17:52:56 - Initialisation complete.
2017-10-31 17:52:56 - Loading configuration information from file 'mypath/hosts.conf'
2017-10-31 17:52:56 - Performing basic connection and environment tests for all machines referenced in jobs.
2017-10-31 17:53:03 - Test connection to 'remotehost' - passed
2017-10-31 17:53:06 - a bytes-like object is required, not 'str'
2017-10-31 17:53:06 - Good bye from Longbow!
2017-10-31 17:53:06 - Check out http://www.hecbiosim.ac.uk/ for other powerful biomolecular simulation software tools.
I downloaded Longbow v1.5.0 via pip and I am currently using python3.6.2
Many thanks!
Jordi
The text was updated successfully, but these errors were encountered: