test: refactor some for loop tests into pytest parametrize

[IronCore864]

This PR tries to refactor (some) test cases with "for" loop or subTest into pytest's parametrize fixture functions.

Only relatively simple changes are made (more on this in section 3).

---

1 TestFiles Changed

• test/test_jujuversion.py
• test/test_log.py
• test/test_pebble.py
• test/test_private.py

---

2 Before/After Comparisons

Num of test cases increased because of parametrize, coverage unchanged.

2.1 test_jujuversion.py

Before

$ python -m tox -e unit -- test/test_jujuversion.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_jujuversion.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 11 items
====================================================================== 11 passed in 0.10s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/jujuversion.py            80     10     48      4    86%   52-58, 62, 66, 76, 80
...
----------------------------------------------------------------------
TOTAL                       6497   4846   2545     32    22%

After

$ python -m tox -e unit -- test/test_jujuversion.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_jujuversion.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 71 items
====================================================================== 71 passed in 0.14s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/jujuversion.py            80     10     48      4    86%   52-58, 62, 66, 76, 80
...
----------------------------------------------------------------------
TOTAL                       6497   4846   2545     32    22%

2.2 test_log.py

Before

$ python -m tox -e unit -- test/test_log.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_log.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 6 items
====================================================================== 6 passed in 0.07s =======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/log.py                    27      4      4      0    81%   67-73
...
----------------------------------------------------------------------
TOTAL                       6497   4875   2545     28    21%
  unit: OK (0.74=setup[0.02]+cmd[0.25,0.47] seconds)
  congratulations :) (0.76 seconds)

After

$ python -m tox -e unit -- test/test_log.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_log.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 10 items
====================================================================== 10 passed in 0.08s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/log.py                    27      4      4      0    81%   67-73
...
----------------------------------------------------------------------
TOTAL                       6497   4875   2545     28    21%
  unit: OK (0.74=setup[0.02]+cmd[0.26,0.46] seconds)
  congratulations :) (0.77 seconds)

2.3 test_pebble.py

Before

$ python -m tox -e unit -- test/test_pebble.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_pebble.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 154 items
===================================================================== 154 passed in 1.15s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/pebble.py               1279     74    431     36    93%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 310, 319, 322->exit, 359, 392, 
...
----------------------------------------------------------------------
TOTAL                       6497   4040   2545     63    34%
  unit: OK (1.83=setup[0.02]+cmd[1.33,0.49] seconds)
  congratulations :) (1.86 seconds)

After

$ python -m tox -e unit -- test/test_pebble.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_pebble.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 164 items
===================================================================== 164 passed in 1.17s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/pebble.py               1279     74    431     36    93%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 310, 319, 322->exit, 359, 392, 419, 452, 536, 572, 605, 616, 661, 678, 726, 862, 938-946, 949, 1011, 1066, 1121, 1222, 1294-1295, 1305, 1518->1520, 1559, 1602, 1653, 1740, 1741->1743, 1806, 1809, 1820-1823, 1830, 2049->2052, 2061, 2130, 2134, 2183, 2185, 2201, 2430, 2452, 2712-2717, 2720-2722, 2865->2869, 2891, 2898, 2901, 2911, 2915->exit, 2924-2925
...
----------------------------------------------------------------------
TOTAL                       6497   4040   2545     63    34%
  unit: OK (1.86=setup[0.02]+cmd[1.35,0.49] seconds)
  congratulations :) (1.88 seconds)

2.4 test_private.py

Before:

$ python -m tox -e unit -- test/test_private.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_private.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 5 items
====================================================================== 5 passed in 0.07s =======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61      0     28      0   100%
ops/_private/yaml.py           9      0      0      0   100%
...
----------------------------------------------------------------------
TOTAL                       6497   4840   2545     28    22%
  unit: OK (0.72=setup[0.02]+cmd[0.24,0.46] seconds)
  congratulations :) (0.75 seconds)

After:

$ python -m tox -e unit -- test/test_private.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_private.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 54 items
====================================================================== 54 passed in 0.10s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
...
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61      0     28      0   100%
ops/_private/yaml.py           9      0      0      0   100%
...
----------------------------------------------------------------------
TOTAL                       6497   4840   2545     28    22%
  unit: OK (0.76=setup[0.02]+cmd[0.28,0.46] seconds)
  congratulations :) (0.79 seconds)

2.5 Total Comparisons

Before

===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 961 items
...
==================================================== 925 passed, 36 skipped, 1 warning in 185.42s (0:03:05) ====================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61      0     28      0   100%
ops/_private/yaml.py           9      0      0      0   100%
ops/charm.py                 643     28    177     17    94%   50-80, 270, 288, 421, 454, 459-461, 475->477, 489, 498-499, 604, 656->660, 671->exit, 673->exit, 677, 682, 745->747, 758, 870, 954, 1167-1168, 1345, 1610-1611, 1777
ops/framework.py             685     54    270     37    89%   62->exit, 64->exit, 65->exit, 66->exit, 123, 126, 164->169, 166->169, 170, 280, 318, 391-392, 534, 559, 562, 597-598, 671->675, 722, 746, 780, 795, 905-907, 927->948, 1062-1066, 1085, 1089, 1094, 1153, 1160, 1181, 1210, 1226, 1257, 1260, 1264, 1268, 1306, 1310, 1314, 1318, 1322, 1326, 1330, 1334, 1338, 1342, 1392, 1396, 1399-1404, 1408, 1412
ops/jujuversion.py            80      6     48      6    91%   53, 57, 62, 66, 76, 80
ops/lib/__init__.py          176     25     77      8    85%   117, 132-134, 142-144, 149-150, 152-153, 158-159, 176-182, 192-196, 222->210, 264->268
ops/log.py                    27      4      4      0    81%   67-73
ops/main.py                  262    117     92      9    49%   56-62, 75-100, 118-122, 132-143, 147, 152-220, 264, 281-301, 307, 331-332, 375-376, 408->410, 455->460, 465-469, 484, 504->509, 546
ops/model.py                1524     77    624     30    94%   302, 314->exit, 316->exit, 430, 480, 790, 806, 838, 921->924, 928, 931, 934, 1041, 1178, 1187-1192, 1275, 1525, 1562, 1623, 1643, 1675, 1787, 1905->1907, 2058, 2061, 2091->2094, 2157->2159, 2247, 2251, 2495, 2552, 2575, 2587, 2597, 2665, 2687, 2769, 2818, 2841, 2858, 2864, 3000, 3024, 3036, 3086, 3135-3136, 3160-3161, 3165-3178, 3233-3243, 3247, 3359, 3390-3393, 3401->3403, 3444->3447, 3489
ops/pebble.py               1279     66    431     36    94%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 310, 319, 322->exit, 359, 392, 419, 452, 536, 572, 605, 616, 661, 678, 726, 862, 949, 1011, 1066, 1121, 1222, 1294-1295, 1305, 1518->1520, 1559, 1602, 1653, 1740, 1741->1743, 1806, 1809, 1820-1823, 1830, 2049->2052, 2061, 2130, 2134, 2183, 2185, 2201, 2430, 2452, 2712-2717, 2720-2722, 2865->2869, 2891, 2898, 2901, 2911, 2915->exit, 2924-2925
ops/storage.py               159      1     30      1    99%   306
ops/testing.py              1580    127    764     69    89%   508->510, 533->535, 562->564, 638->637, 870->872, 885->888, 914->922, 923->927, 936, 953, 986, 992, 1044, 1059->exit, 1073, 1192->exit, 1243, 1247->1252, 1264->1262, 1329->1327, 1635, 1638->1640, 1665, 1670, 1758-1760, 1908, 1916, 2038, 2123, 2147, 2152-2158, 2161, 2256-2257, 2298, 2313-2332, 2353->2358, 2402, 2405, 2459, 2518-2520, 2528, 2532, 2535, 2569, 2581, 2591->exit, 2593, 2617, 2638->2637, 2668-2671, 2692, 2699-2702, 2704-2707, 2709, 2711-2714, 2770, 2781, 2786, 2846, 2862, 2868, 2876, 2916, 2919, 2925, 2928, 2931, 2947, 2994-3008, 3013, 3021, 3028, 3048->3037, 3053, 3249-3250, 3329-3330, 3340, 3386, 3406-3430, 3433, 3449-3450, 3509, 3523, 3525-3527, 3530, 3547, 3549
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6497    505   2545    213    90%
  unit: OK (186.33=setup[0.02]+cmd[185.68,0.63] seconds)
  congratulations :) (186.36 seconds)

After

$ python -m tox -e unit
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator2/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator2
collected 1084 items
...
=================================================== 1048 passed, 36 skipped, 1 warning in 190.34s (0:03:10) ====================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61      0     28      0   100%
ops/_private/yaml.py           9      0      0      0   100%
ops/charm.py                 643     28    177     17    94%   50-80, 270, 288, 421, 454, 459-461, 475->477, 489, 498-499, 604, 656->660, 671->exit, 673->exit, 677, 682, 745->747, 758, 870, 954, 1167-1168, 1345, 1610-1611, 1777
ops/framework.py             685     54    270     37    89%   62->exit, 64->exit, 65->exit, 66->exit, 123, 126, 164->169, 166->169, 170, 280, 318, 391-392, 534, 559, 562, 597-598, 671->675, 722, 746, 780, 795, 905-907, 927->948, 1062-1066, 1085, 1089, 1094, 1153, 1160, 1181, 1210, 1226, 1257, 1260, 1264, 1268, 1306, 1310, 1314, 1318, 1322, 1326, 1330, 1334, 1338, 1342, 1392, 1396, 1399-1404, 1408, 1412
ops/jujuversion.py            80      6     48      6    91%   53, 57, 62, 66, 76, 80
ops/lib/__init__.py          176     25     77      8    85%   117, 132-134, 142-144, 149-150, 152-153, 158-159, 176-182, 192-196, 222->210, 264->268
ops/log.py                    27      4      4      0    81%   67-73
ops/main.py                  262    117     92      9    49%   56-62, 75-100, 118-122, 132-143, 147, 152-220, 264, 281-301, 307, 331-332, 375-376, 408->410, 455->460, 465-469, 484, 504->509, 546
ops/model.py                1524     77    624     30    94%   302, 314->exit, 316->exit, 430, 480, 790, 806, 838, 921->924, 928, 931, 934, 1041, 1178, 1187-1192, 1275, 1525, 1562, 1623, 1643, 1675, 1787, 1905->1907, 2058, 2061, 2091->2094, 2157->2159, 2247, 2251, 2495, 2552, 2575, 2587, 2597, 2665, 2687, 2769, 2818, 2841, 2858, 2864, 3000, 3024, 3036, 3086, 3135-3136, 3160-3161, 3165-3178, 3233-3243, 3247, 3359, 3390-3393, 3401->3403, 3444->3447, 3489
ops/pebble.py               1279     66    431     36    94%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 310, 319, 322->exit, 359, 392, 419, 452, 536, 572, 605, 616, 661, 678, 726, 862, 949, 1011, 1066, 1121, 1222, 1294-1295, 1305, 1518->1520, 1559, 1602, 1653, 1740, 1741->1743, 1806, 1809, 1820-1823, 1830, 2049->2052, 2061, 2130, 2134, 2183, 2185, 2201, 2430, 2452, 2712-2717, 2720-2722, 2865->2869, 2891, 2898, 2901, 2911, 2915->exit, 2924-2925
ops/storage.py               159      1     30      1    99%   306
ops/testing.py              1580    127    764     69    89%   508->510, 533->535, 562->564, 638->637, 870->872, 885->888, 914->922, 923->927, 936, 953, 986, 992, 1044, 1059->exit, 1073, 1192->exit, 1243, 1247->1252, 1264->1262, 1329->1327, 1635, 1638->1640, 1665, 1670, 1758-1760, 1908, 1916, 2038, 2123, 2147, 2152-2158, 2161, 2256-2257, 2298, 2313-2332, 2353->2358, 2402, 2405, 2459, 2518-2520, 2528, 2532, 2535, 2569, 2581, 2591->exit, 2593, 2617, 2638->2637, 2668-2671, 2692, 2699-2702, 2704-2707, 2709, 2711-2714, 2770, 2781, 2786, 2846, 2862, 2868, 2876, 2916, 2919, 2925, 2928, 2931, 2947, 2994-3008, 3013, 3021, 3028, 3048->3037, 3053, 3249-3250, 3329-3330, 3340, 3386, 3406-3430, 3433, 3449-3450, 3509, 3523, 3525-3527, 3530, 3547, 3549
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6497    505   2545    213    90%
  unit: OK (191.21=setup[0.02]+cmd[190.58,0.61] seconds)
  congratulations :) (191.24 seconds)

---

3 TODO

Need more research on other test files (similar to test_logs.py but more complicated), will report back later.

[benhoyt]

Looks pretty good to me so far, thanks!

[tonyandrewmeyer]

Looks good, thanks!