Setting the message model dir does not work

[volker-krey]

Setting the message model dir via the command message-ix config set "message model dir" /path/for/model/files does not work for me. I am using the conda based message_ix installation and wanted to use the GAMS code in my working copy of the message_ix repository to allow making changes to the GAMS code.

Problem description

I ran the following command in the Anaconda command line so that message model dir points to the model folder of my working copy.

message-ix config set "message model dir" C:/Users/.../Documents/git/message_ix/message_ix/model/

I also tested the message-ix copy-model command which in principle did most of what I was expecting it to do with the exception of copying the files directly sitting in the model directory (e.g., MESSAGE_run.gms) which would be important as modifications could also be relevant there.

message-ix copy-model C:/Users/.../Documents/TEMP/model/

Versions

Below is the version information for ixmp and message_ix:

(message) C:\>pip show ixmp
Name: ixmp
Version: 3.2.0
Summary: ix modeling platform
Home-page: https://github.com/iiasa/ixmp
Author: IIASA IIASA Energy, Climate, and Environment (ECE) program
Author-email: message_ix@iiasa.ac.at
License: Apache
Location: c:\users\...\anaconda3\envs\message\lib\site-packages
Requires: sparse, PyYAML, pandas, JPype1, xarray, click, pint, dask, openpyxl, graphviz
Required-by: message-ix

(message) C:\>pip show message-ix
Name: message-ix
Version: 3.2.0
Summary: the MESSAGEix integrated assessment model
Home-page: http://github.com/iiasa/message_ix
Author: IIASA Energy, Climate, and Environment (ECE) Program
Author-email: message_ix@iiasa.ac.at
License: Apache
Location: c:\users\...\anaconda3\envs\message\lib\site-packages
Requires: PyYAML, ixmp, click, numpy, setuptools, pandas
Required-by: message-data

[khaeru]

Thanks for this report!

In order to reproduce and be sure this is fixed, can you say specifically: how does it "not work", or what are the symptoms?

For example, I imagine it could be one of:

1. The CLI command message-ix config set … produces an error.
2. The CLI command runs fine (thus, not (1)), but then running certain code (e.g. solving a MESSAGEix scenario) produces some error.
3. The CLI command and code (e.g. solving a scenario) both run without error, but the setting doesn't have the intended effect, i.e. the configured alternate location for the GAMS files is not used, and instead the (default) model files packaged with the code are used.

If (3), can you also give the contents of ~.local/share/ixmp/config.json after running the command?

[volker-krey]

The command does not produce any error (with exception of a warning related pandas package - see below), but it has no effect. In other words, the original GAMS code in the Anaconda installation keeps being used (with the gdx, log and lst files materializing in the model folder of the Anaconda installation).

Warning message:

C:\Users\...\Anaconda3\envs\message\lib\site-packages\message_ix\testing\__init__.py:48: FutureWarning: set_axis currently defaults to operating inplace.
This will change in a future version of pandas, use inplace=True to avoid this warning.
  columns.split(), axis=1

[volker-krey]

My config.json file actually includes a line that indicates that running the message model dir was set correctly, but as mentioned it does not seem to have an effect:

"message model dir": "C:\\Users\\...\\Documents\\git\\message_ix\\message_ix\\model"

PS: Note that in an earlier version of this message it said that there was no config.json file, since I was looking at the wrong .local directory.

[khaeru]

Thanks for the information. This should be small, so I will add it to my list before the 3.3 release.

[khaeru]

Sorry, I think this needs some further info to diagnose:

1. Do three periods ... literally occur in the file path?

   "message model dir": "C:\\Users\\...\\Documents\\git\\message_ix\\message_ix\\model"
   

   I know that Windows understands . and .. as on *nix, but I'm not aware this is a proper path. I would imagine that this literal value would cause an actual error if message-ix tried to use it.
2. Where is config.json located?
3. What are the outputs of message-ix show-versions and python -c "import ixmp; print(ixmp.config.path)"?

[khaeru]

#416 introduced some tests for this functionality that seem to work, including on Windows. Remove the milestone until we have the info needed to diagnose.

[volker-krey]

@khaeru, thanks. Related to the questions raised above:

1. In reality, the path includes the user account name on my machine which I replaced for security reasons with three dots. So this is actually a valid path.
2. The config.json sits in a custom location (C:\Users...\Documents\git\ixmp\config - three dots again replacing login) which is set via the IXMP_DATA environment variable.
3. see below.

message-ix show-versions

ixmp:        3.2.0
message_ix:  3.2.0
message_data: 2020.6.21.dev181+g5d305dd
     1d1aace4 (HEAD -> material_intensity_electricity, origin/material_intensity_electricity) Merge branch 'material_intensity_electricity' of https://github.com/volker-krey/message_data into material_intensity_electricity

click:       8.0.0
dask:        2021.05.0
graphviz:    0.14.2
jpype:       1.2.1
… JVM path:  C:\Users\...\Anaconda3\envs\message\Library\bin\server\jvm.dll
openpyxl:    3.0.7
pandas:      1.2.4
pint:        0.17
xarray:      0.18.0
yaml:        5.4.1

iam_units:   installed
jupyter:     installed
matplotlib:  3.0.2
plotnine:    0.8.0
pyam:        0.11.0

GAMS:        25.1.3

python:      3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 15:37:01) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS:          Windows
OS-release:  10
machine:     AMD64
processor:   Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder:   little
LC_ALL:      None
LANG:        None
LOCALE:      None.None

python -c "import ixmp; print(ixmp.config.path)"
C:\Users\...\Documents\git\ixmp\config\config.json

I am realizing that under message_data an old working copy that I am not using anymore is listed. Hope that is not the reason for the problem I am having.