Support for upgrading the exporter config file

Signed-off-by: Andreas Maier <maiera@de.ibm.com>

changes/571.feature.rst

@@ -0,0 +1,6 @@
+Support for upgrading a config file (= HMC credentials file) from exporter
+version 1.x to the current version. By default, the exporter configuration is
+upgraded internally without persisting the changes. A new '--upgrade-config'
+command line option can be used to upgrade the exporter config file. This
+required using the 'ruamel.yaml' Python package, in order to preserve comments
+in the exporter config file.

dev-requirements.txt

@@ -21,7 +21,7 @@ safety>=3.0.1
 safety-schemas>=0.0.2
 # TODO: Change to dparse 0.6.4 once released
 dparse>=0.6.4b0
-ruamel.yaml>=0.17.21
+# ruamel.yaml is used by safety and is covered in requirements.txt
 click>=8.0.2
 Authlib>=1.2.0
 marshmallow>=3.15.0
@@ -33,6 +33,7 @@ psutil>=6.0.0
 
 # Bandit checker
 bandit>=1.7.8
+# PyYAML is used by bandit and is covered in requirements.txt
 
 # Unit test (imports into testcases):
 pytest>=6.2.5

docs/usage.rst

@@ -111,6 +111,9 @@ The ``zhmc_prometheus_exporter`` command supports the following arguments:
 
       --version             show versions of exporter and zhmcclient library and exit
 
+      --upgrade-config      upgrade the exporter config file to the current version of the exporter
+                            and exit
+
       --help-config         show help for exporter config file and exit
 
 
@@ -928,85 +931,15 @@ Migration of exporter config file to version 2 format
 The exporter versions 1.x supported the version 1 format for the exporter config
 file and referred to it as the "HMC credentials file". If you have been using
 an exporter version 1.x, you should migrate your "HMC credentials file" to
-the version 2 format. The version 1 format is deprecated and support for it
-will be removed in a future version of the exporter.
-
-Migration to the version 2 format is done as follows:
-
-* Add a 'version' item:
-
-  .. code-block:: yaml
-
-      version: 2
-
-* Change the old 'metrics' item to the new 'hmc' item:
-
-  'metrics' item from version 1 format:
-
-  .. code-block:: yaml
-
-      metrics:
-        hmc: 9.10.11.12
-        userid: userid
-        password: password
-        verify_cert: true
-
-  'hmc' item from version 2 format:
-
-  .. code-block:: yaml
-
-      hmc:                     # changed
-        host: 9.10.11.12       # changed
-        userid: userid
-        password: password
-        verify_cert: true
-
-* Add a 'metric_groups' item to specify which metric groups to export:
-
-  .. code-block:: yaml
-
-      metric_groups:
-        # Available for CPCs in classic mode
-        cpc-usage-overview:
-          export: true
-        logical-partition-usage:
-          export: true
-        channel-usage:
-          export: true
-        crypto-usage:
-          export: true
-        flash-memory-usage:
-          export: true
-        roce-usage:
-          export: true
-        logical-partition-resource:
-          export: true
-        # Available for CPCs in DPM mode
-        dpm-system-usage-overview:
-          export: true
-        partition-usage:
-          export: true
-        adapter-usage:
-          export: true
-        network-physical-adapter-port:
-          export: true
-        partition-attached-network-interface:
-          export: true
-        partition-resource:
-          export: true
-        storagegroup-resource:
-          export: true
-        storagevolume-resource:
-          export: true
-        # Available for CPCs in any mode
-        zcpc-environmentals-and-power:
-          export: true
-        zcpc-processor-usage:
-          export: true
-        environmental-power-status:
-          export: true
-        cpc-resource:
-          export: true
+the version 2 format. The version 1 format is still supported but it is
+internally upgraded to the current version when using the exporter, without
+persisting the changes to the file.
+
+The '--upgrade-config'  command line option can be used to upgrade the
+exporter config file to the current version. When doing that, YAML comments
+are preserved, except when they are preceding or following any items that are
+removed as part of the upgrade (e.g. the 'metrics' item in a version 1 file).
+
 
 Sample exporter config file
 ---------------------------

minimum-constraints-develop.txt

@@ -20,7 +20,7 @@ safety==3.0.1
 safety-schemas==0.0.2
 # TODO: Change to dparse 0.6.4 once released
 dparse==0.6.4b0
-ruamel.yaml==0.17.21
+# ruamel.yaml is used by safety and is covered in minimum-constraints-install.txt
 click==8.0.2
 Authlib==1.3.1
 marshmallow==3.15.0
@@ -32,6 +32,7 @@ psutil==6.0.0
 
 # Bandit checker
 bandit==1.7.8
+# PyYAML is used by bandit and is covered in minimum-constraints-install.txt
 
 # Unit test (imports into testcases):
 pytest==6.2.5

minimum-constraints-install.txt

@@ -26,12 +26,14 @@ urllib3==1.26.19
 jsonschema==3.2.0
 six==1.16.0
 Jinja2==3.1.4
-# PyYAML is also used by dparse
-PyYAML==5.3.1
+ruamel.yaml==0.18.6
 
 
 # Indirect dependencies for install that are needed for some reason (must be consistent with requirements.txt)
 
+# PyYAML is used by zhmcclient 1.17
+PyYAML==5.3.1
+
 pyrsistent==0.18.1
 
 

requirements.txt

@@ -21,7 +21,12 @@ zhmcclient>=1.17.0
 urllib3>=1.26.19
 jsonschema>=3.2.0
 Jinja2>=3.1.4
+ruamel.yaml>=0.18.6
 
+
+# Indirect dependencies for install that are needed for some reason (must be consistent with minimum-constraints-install.txt)
+
+# PyYAML is used by zhmcclient 1.17
 # PyYAML 5.3.x has wheel archives for Python 2.7, 3.5 - 3.9
 # PyYAML 5.4.x has wheel archives for Python 2.7, 3.6 - 3.9
 # PyYAML 6.0.0 has wheel archives for Python 3.6 - 3.11
@@ -31,9 +36,6 @@ Jinja2>=3.1.4
 PyYAML>=5.3.1,!=5.4.0,!=5.4.1; python_version <= '3.11'
 PyYAML>=5.3.1,!=5.4.0,!=5.4.1,!=6.0.0; python_version >= '3.12'
 
-
-# Indirect dependencies for install that are needed for some reason (must be consistent with minimum-constraints-install.txt)
-
 # pyrsistent is used by jsonschema 3.x (no longer by jsonschema 4.x)
 # pyrsistent 0.15.0 fixes import errors on Python>=3.10, but only 0.18.1 has
 #   Python 3.10 support (accordong to the change log).