Read GAMS CPLEX solver options for MESSAGE from user config

[khaeru]

This file adds a new recognized key, "message model options", in the user configuration file (~/.local/share/ixmp/config.json or similar), that corresponds to passing options to Scenario.solve(…) (docs).

So, for instance, with the following in config.json:

{
  "my-platform": {"backend": "jdbc", "etc": "etc"},
  "...": "...",
  "message solve options": {"lpmethod": 4},
}

…then:

s.solve()

is updated to have the same effect as:

s.solve(solve_options={"lpmethod": 4})

Also:

• Include Python version 3.10 and remove 3.6 in CI #532
• Setting the message model dir does not work  ixmp#415 fixed the issue underlying “message model dir” not working as described here.

How to review

• Read the diff and note that the CI checks all pass.

PR checklist

• Open a corresponding PR upstream (in ixmp) to address the FIXME in the added comment → Simplify & improve configuration ixmp#435.
• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update release notes.

[khaeru]

FYI @gidden @OFR-IIASA

[codecov]

Codecov Report

Merging #557 (1bde1bf) into main (1df3619) will decrease coverage by 1.3%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##            main    #557     +/-   ##
=======================================
- Coverage   93.8%   92.5%   -1.4%     
=======================================
  Files         40      41      +1     
  Lines       3057    3153     +96     
=======================================
+ Hits        2870    2919     +49     
- Misses       187     234     +47     

Impacted Files	Coverage Δ
message_ix/__init__.py
message_ix/models.py
message_ix/tests/test_tutorials.py
message_ix/tests/test_legacy_version.py
message_ix/cli.py
message_ix/reporting/pyam.py
message_ix/tests/test_feature_price_commodity.py
message_ix/tests/test_reporting.py
message_ix/tests/test_feature_duration_time.py
message_ix/util/tutorial.py
... and 71 more

[gidden]

Thanks @khaeru! Seems like the right solution.

[khaeru]

This is also affected by #559.

[khaeru]

So @LauWien if I understand the description at #566 (comment), then the issue in .tools.add_year (#559) should go away now that pandas 1.4.1 (I guess, different from 1.4.0) is used and we are no longer using Python 3.6?

[khaeru]

It looks like there were several regressions related to pd.DataFrame.loc fixed in pandas 1.4.1: https://pandas.pydata.org/pandas-docs/stable/whatsnew/v1.4.1.html#fixed-regressions. Maybe one of them is the one we were seeing 🤞🏾

[LauWien]

So @LauWien if I understand the description at #566 (comment), then the issue in .tools.add_year (#559) should go away now that pandas 1.4.1 (I guess, different from 1.4.0) is used and we are no longer using Python 3.6?

Yes, I think so. I ran the tests here now, and "only" the test test_tutorial[R_austria] is failing on ubuntu-latest so far. Lets see 🙈 :D

[LauWien]

test_tutorial[R_austria] is failing on ubuntu-latest so far

Just a "time out" error.

E           nbclient.exceptions.CellTimeoutError: A cell timed out while it was being executed, after 10 seconds.
E           The message was: Cell execution timed out.
E           Here is a preview of the cell contents:
E           -------------------
E           year_pairs = as.matrix(expand.grid(horizon,horizon) %>%
E                                    rowwise() %>%
E                                    filter(Var2 >= Var1) %>%
E                                    arrange(Var1))
E           vintage_years <- year_pairs[,1]
E           act_years <- year_pairs[,1]
E           -------------------

[khaeru]

Just a "time out" error.

A little weird as (1) it's occurring on all Python versions and (2) that R code isn't very complex.

[khaeru]

• Add, expand, or update documentation.
• Update release notes.

I still need to do these items, so will try them now and then see if the Jupyter R test still fails when I push again.

[khaeru]

see if the Jupyter R test still fails

It did. In 1bde1bf I added an exclusion, replacing an older one that is no longer necessary because we no longer test on Python 3.6.

[khaeru]

If checks pass, please approve and merge using a merge commit.

[LauWien]

If checks pass, please approve and merge using a merge commit.

Will do :) Many thanks!

[LauWien]

If checks pass

Hm, the coverage will decrease by -1.4%.

[khaeru]

Codecov seems inconsistent, e.g. if we look here https://app.codecov.io/gh/iiasa/message_ix/branch/feature%2Fsolve-args-config —the number of missed lines is still 187, different from the comment above.

Sometimes this takes a few seconds or minutes to get updated.

[khaeru]

@LauWien the discrepancy might also be due to this update to a newer version of the Codecov action.

Compare

• https://app.codecov.io/gh/iiasa/message_ix/branch/main at the bottom, the code is in the directory "message_ix" (using the older version of the action)
• https://app.codecov.io/gh/iiasa/message_ix/branch/feature%2Fsolve-args-config at the bottom, we instead see "message_ix/message_ix" (using the newer version)

The difference of 47 lines corresponds to .testing.nightly, so I think this might resolve itself at the next nightly test run.

[LauWien]

Ah, yes, I'll then merge!