UnicodeEncodeError: 'ascii' codec can't encode characters in position ...

[prvst]

Hello, my runs keep crashing with the same error message. Could you please help me figure it out? I'm running it on idXML files from either Comet and MSFragger (pepXML -> idXML).

Thanks.

ms2pip.core // Processing spectra and peptides...
ms2rescore.feature_generators.deeplc // Adding DeepLC-derived features to PSMs.
ms2rescore.feature_generators.deeplc // Running DeepLC for PSMs from run (1/1): `<filename>`...
ms2rescore.__main__ // 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/ms2rescore/__main__.py", line 237, in main
    rescore(configuration=config)
  File "/usr/local/lib/python3.11/site-packages/ms2rescore/core.py", line 79, in rescore
    fgen.add_features(psm_list)
  File "/usr/local/lib/python3.11/site-packages/ms2rescore/feature_generators/deeplc.py", line 155, in add_features
    self.deeplc_predictor.calibrate_preds(psm_list_calibration)
  File "/usr/local/lib/python3.11/site-packages/deeplc/deeplc.py", line 1173, in calibrate_preds
    calibrate_output = self.calibrate_preds_func_pygam(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/deeplc/deeplc.py", line 830, in calibrate_preds_func_pygam
    predicted_tr = self.make_preds(psm_list, calibrate=False, mod_name=mod_name)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/deeplc/deeplc.py", line 750, in make_preds
    ret_preds = self.make_preds_core(
                ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/deeplc/deeplc.py", line 621, in make_preds_core
    ret_preds = mod.predict(
                ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.11/site-packages/keras/src/utils/io_utils.py", line 98, in print_msg
    sys.stdout.write(message)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 20-21: ordinal not in range(128)

[RalfG]

Hi @prvst!

Thanks for reporting! After quite some digging, we discovered that the problem was an incompatibility between the new Keras/Tensorflow progress bars and how MS²Rescore suppresses Tensorflow logs that do not use the logging module.

We will fix this in the next MS²Rescore release. For now, running MS²Rescore with log_level set to debug circumvents the issue.

Best,
Ralf

---

For future technical reference, the issue was somewhat related to keras-team/keras#19386. We used with contextlib.redirect_stdout(open(os.devnull, "w")), which opens os.devnull with the ASCII codec, while Keras uses UTF-8 symbols for the progress bars. Opening os.devnull with UTF-8 fixes the problem.