Support for GAMS programs where the main file is not in the root folder

[spine-o-bot]

In GitLab by @ererkka on Mar 30, 2020, 11:21

Status and need

Some GAMS programs have a delicate directory structure, and the main file might not be in the root folder of the project. In case of Balmorel, the project is structured as follows (not all files are show):

Balmorel
├───base
│   ├───addons
│   ├───auxils
│   ├───bui
│   ├───data
|   |      (this folder has the input data files)
│   ├───documentation
│   ├───logerror
│   │   └───logerinc
│   ├───model
|   |      Balmorel.gms   *This is the main file*
│   └───output
│       ├───economic
│       ├───inputout
│       ├───printout
│       │   └───printinc
│       └───temp
└───simex

Setting this up using the Toolbox interface is possible, but problems start when trying to execute in a work folder. Toolbox takes the folder Balmorel/base/model as the main_dir and copies everything from there into a temporary work directory root (i.e. something like /balmorel__xxxxxxxxxxxxxxxxxx__toolbox). But some files and the data should be outside of this folder and on the same or above level in hierarchy, so they end up in wrong places and the GAMS program does not find them.

My work-around

My temporary solution is to edit the tool specification file by hand to list the source files like this:

{
    . . .
    "includes": [
        "base/model/Balmorel.gms",
        "base/model/Balmorelbb4.inc",
        "base/model/Balmorelbb4.sim",
        "base/model/bb4datainc.inc",
        "base/model/bb123.sim",
        "base/model/syminput.inc",
        "base/model/symoutput.inc",
        "base/logerror/logerinc/balbase1.mss",
        "base/logerror/logerinc/balbase2.mss",
        "base/logerror/logerinc/balbase3.mss",
        "base/logerror/logerinc/balbase4.mss",
        "base/logerror/logerinc/error1.inc",
        "base/logerror/logerinc/error2.inc",
        "base/logerror/logerinc/error3.inc",
        "base/logerror/logerinc/error4.inc",
        "base/output/OUTPUT_SUMMARY.inc",
        "base/output/printout/printinc/EG3_CY.INC",
        "base/output/printout/printinc/EHMCO2qRT.inc",
        "base/output/printout/printinc/aep_yna.inc",
        . . .
    ],
    "includes_main_path": "Balmorel"
    . . .
}

The specification file is stored in the Toolbox project folder alongside with the directory Balmorel which has the source code. Now files are copied into their right places into a temp working dir.

GAMS needs to have the current dir set to where the main program file is located. Toolbox does that by setting the CurDir GAMS command line argument. I’ve made the following change to make this work:

--- a/spinetoolbox/tool_specifications.py
+++ b/spinetoolbox/tool_specifications.py
@@ -400,7 +400,7 @@ class GAMSTool(ToolSpecification):
         Args:
             basedir (str): the path to the directory where the instance will run
         """
-        return GAMSToolInstance(self, basedir, self._settings, self._logger)
+        return GAMSToolInstance(self, os.path.join(basedir, self.main_dir), self._settings, self._logger)
 

This will now pass the temp workdir (basedir) joined with the main dir (self.main_dir) to the CurDir argument. Something like this:

 gams Balmorel.gms CurDir=<my temp>/balmorel__xxxxxxxxxxxxxxxxxx__toolbox/base/model

and GAMS works nicely.

My proposal

Change the main file selection in the Tool spec. editor into source code folder selector. In the above case that would be folder Balmorel. In addition we need a main file selection box, to set the main program file explicitly, and it will be stored as relative to the source code folder. Other source files will be relative to that folder too.

Benefit: This would probably allow extending (easily?) to support Git repository paths as the source code folder. The user just needs to know the internal structure to insert the information about main file etc.

[spine-o-bot]

In GitLab by @ererkka on Mar 30, 2020, 13:09

changed the description

[spine-o-bot]

In GitLab by @ererkka on Mar 30, 2020, 13:10

changed the description

[spine-o-bot]

In GitLab by @PekkaSavolainen on Apr 9, 2020, 09:19

assigned to @PekkaSavolainen

[spine-o-bot]

In GitLab by @PekkaSavolainen on Apr 9, 2020, 17:53

mentioned in commit c30efec74a3b653fd2652ec9db93ca2e7ddc282c

[jkiviluo]

I think Erkka's solution is very nice.

[ptsavol]

Done. See PR spine-tools/spine-items#238