Python 3 finalization tasks

[mliu49]

Once #1724 is merged, the bulk of the Python 3 transition will be completed. However, there are final testing and clean up tasks which will still need to be done. The purpose of this issue is to keep track of these tasks and make sure that they are done before releasing RMG 3.0.

To do:

• Remove all __future__ imports Remove future imports and py2/py3 compatibility imports #1730
• Update all IPython notebooks Update IPython notebooks #1735
• Update RMG-database scripts and notebooks Update RMG-database scripts and notebooks to Python 3 RMG-database#364
• Remove dde models from RMG-database Remove DDE models RMG-database#358
• Update documentation as needed @hwpang (particularly regarding installation @mliu49 and @amarkpayne ) Update documentation from py2 to py3 #1767
• Update global uncertainty analysis (transition to MUQ 2) Update global uncertainty module to work with MUQ 2 #1738

Testing:

• Run a pdep job @alongd
• Run a job with ml thermo @yunsiechung
• Run a job with QMTP @agnes (debug conformer issue @dranasinghe and @alongd )
• Run a catalysis job @maforsuelo
• Run a parallel job - natural gas combustion case for 3d 20h no error but see for warning messages below
• Run a liquid job @hwpang
• Run a large pdep network in Arkane @alongd
• Run an isotopes job @goldmanm
• Run sensitivity and local uncertainty analysis @oscarwumit
• Test all scripts @yunsiechung
• Test all IPython notebooks

Note that some of jobs are run by RMG-tests, but the point here is to run a longer job and ideally a case that is important to you.

Please comment (or edit) if you have suggestions for other items to add to either list.

[ajocher]

Running the examples/rmg/1,3-hexadiene case with or without pdep but both times with QMTP on the RMG-server: after 6:52 min run time this error happens

File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmg.py", line 107, in
main()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmg.py", line 101, in main
rmg.execute(**kwargs)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/rmg/main.py", line 651, in execute
self.initialize(**kwargs)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/rmg/main.py", line 486, in initialize
self.load_database()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/rmg/main.py", line 410, in load_database
family.add_rules_from_training(thermo_database=self.database.thermo)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/data/kinetics/family.py", line 1231, in add_rules_from_training
reactant.thermo = thermo_database.get_thermo_data(reactant, training_set=True)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/data/thermo.py", line 1244, in get_thermo_data
tdata = self.estimate_radical_thermo_via_hbi(molecule, quantum_mechanics.get_thermo_data)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/data/thermo.py", line 1841, in estimate_radical_thermo_via_hbi
thermo_data_sat = stable_thermo_estimator(saturated_struct)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/main.py", line 217, in get_thermo_data
thermo0 = qm_molecule_calculator.generate_thermo_data()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/molecule.py", line 362, in generate_thermo_data
self.qm_data = self.generate_qm_data()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/mopac.py", line 303, in generate_qm_data
self.create_geometry()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/molecule.py", line 328, in create_geometry
self.geometry.generate_rdkit_geometries()
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/molecule.py", line 138, in generate_rdkit_geometries
self.save_coordinates_from_rdmol(rdmol, min_e_id, rdatom_idx)
File "/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/qm/molecule.py", line 176, in save_coordinates_from_rdmol
point = rdmol.GetConformer(min_e_id).GetAtomPosition(atom.sorting_label)
ValueError: Bad Conformer Id

[mliu49]

Could you double-check this example on v2.4.1? I got the error on both master and the v2.5.0 branch.

Initial investigation suggests issues with RDKit conformer generation. The molecule this breaks on is C1=C2CCC(C1)C2, and RDKit fails to generate any conformers for it for some reason.

[ajocher]

RMG v2.4.1 fails with the same error.
The current git HEAD for RMG-Py is:
af0ef48
Tue Jul 23 14:48:04 2019 -0400
The current git HEAD for RMG-database is:
d52ebab
Thu Jul 18 10:30:25 2019 -0400

RMG v2.4.0 fails with the same error using the v2.4.1 rmg_env but works with v2.4.0 rmg_env
The current git HEAD for RMG-Py is:
caebd48
Fri Jun 14 18:30:27 2019 -0400
The current git HEAD for RMG-database is:
08b8214
Fri Jun 14 18:15:05 2019 -0400

The v2.4.1 environment requests "-rdkit >=2018" while the v2.4.0 environment uses rdkitVersion '2016.03.4'. Not sure what good fix would be.

[ajocher]

While running the parallel case the following warning messages were printed

/home/ajocher/codes/RMG/miniconda3/envs/rmg_env/lib/python3.7/site-packages/scipy/optimize/optimize.py:1826: LinAlgWarning: Ill-conditioned matrix (rcond=9.79183e-18): result may not be accurate.
fu = func(x, *args)
/home/ajocher/codes/RMG/miniconda3/envs/rmg_env/lib/python3.7/site-packages/scipy/optimize/optimize.py:1826: LinAlgWarning: Ill-conditioned matrix (rcond=5.9437e-17): result may not be accurate.
fu = func(x, *args)
/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/rmg/main.py:793: RuntimeWarning: invalid value encountered in true_divide
conditions=self.rmg_memories[index].get_cond()
/home/ajocher/codes/RMG/RMG3/RMG-Py/rmgpy/rmg/main.py:793: RuntimeWarning: divide by zero encountered in true_divide
conditions=self.rmg_memories[index].get_cond()
/home/ajocher/codes/RMG/miniconda3/envs/rmg_env/lib/python3.7/site-packages/scipy/optimize/optimize.py:1826: LinAlgWarning: Ill-conditioned matrix (rcond=5.11164e-17): result may not be accurate.
fu = func(x, *args)

[yunsiechung]

Arkane does not generate conformer and thermo any longer in output.py for thermo job.

Running examples/arkane/species/Benzyl/input.py will produce the output.py that is missing conformer and thermo

[yunsiechung]

I checked ml thermo by:

• running rmg minimal_ml example and checking that the thermo of the species that passed the ml estimating criteria was estimated from ml
• making sure the unittests for ml thermo are still there and they all passed.

Ml thermo looks fine.

[github-actions]

This issue is being automatically marked as stale because it has not received any interaction in the last 90 days. Please leave a comment if this is still a relevant issue, otherwise it will automatically be closed in 30 days.