suite-state command forward compat.

[hjoliver]

Address #5236
Sibling of #5237

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg and conda-environment.yml.
• Tests are included (or explain why tests are not needed).
• CHANGES.md entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[hjoliver]

Not sure if we really need this one (it'd be nice to avoid yet another final Cylc 7 release).

#5237 is the critical one, for Cylc 8 research users triggering tasks off of Cylc 7 operational suites that haven't been migrated yet.

[dwsutherland]

For operations, this is important for migration, otherwise we'll be forced to upgrade downstream first.. Which we could do I suppose...

[hjoliver]

Can you give this a quick review and test @dwsutherland ?

(Since we need it soon!)

[hjoliver]

(New tests are not needed at this point, for Cylc 7 IMO)

[MetRonnie]

Missing the xtrigger part here?

cylc-flow/lib/cylc/xtriggers/suite_state.py

Line 49 in e60e8f7

fmt = checker.get_remote_point_format()

[hjoliver]

Good spot @MetRonnie, thanks.

Turns out I hadn't quite finished this off - another change was need to handle the change in task_outputs table format. (Well, 11 months is a long time...)

[hjoliver]

Testing:

Run with Cylc 8:

[scheduler]
    cycle point format = %Y
[scheduling]
    initial cycle point = 2010
    final cycle point = 2012
   [[graph]]
      P1Y = "foo"
[runtime]
   [[foo]]
      script = "cylc message x"
      [[[outputs]]]
          x = x

Now with Cylc 7, both of these work:

cylc-7 $ cylc suite-state c8/runN --task=foo --point=2011 --message=x
polling for 'output: x': satisfied

cylc-7 $ cylc suite-state c8/runN --task=foo --point=2011 --status=succeeded
polling for 'succeeded': satisfied

And to test xtriggers polling the above Cylc 8 workflow:

# suite.rc
[cylc]
  cycle point format = %Y
[scheduling]
    initial cycle point = 2010
    final cycle point = 2012
    [[xtriggers]]
         x1 = suite_state(suite=c8/runN, task=foo, point=%(point)s, message=x)
         x2 = suite_state(suite=c8/runN, task=foo, point=%(point)s, status=succeeded)
   [[dependencies]]
        [[[P1Y]]]
           graph = """
                @x1=> foo
                @x2 => foo
           """
[runtime]
    [[foo]]

[hjoliver]

Should be good to go now.

[MetRonnie]

Are you saying removing the .values() makes it work for Cylc 8 DBs? Surely that breaks Cylc 7?

[MetRonnie]

Or was that just a mistake?

Suggested change

	# This works for Cylc 7 and Cylc 8 outputs table format:
	return any(
	message == value
	for outputs_str, in res
	for value in json.loads(outputs_str)
	)
	# This works for Cylc 7 and Cylc 8 outputs table format:
	return any(
	message == value
	for outputs_str, in res
	for value in json.loads(outputs_str).values()
	)

[hjoliver]

Damn it. I owe you a chocolate fish 🐟

It wasn't a mistake - I took it from master, and I tested it.
However, I tested it with our classic shortcut x = x outputs - i.e. with message and label the same 🤦

So this is a bug on master too. Patch coming...

[MetRonnie]

So this is a bug on master too.

You mean, only a bug on master/8.x? (Though nearly introduced to Cylc 7 in this PR)

[dwsutherland]

Works for me:

[sutherlanddw@w-ec-admin01 ~]$ cylc --version
7.9.2
[sutherlanddw@w-ec-admin01 ~]$ cylc suite-state download/run1 --run-dir=/home/sutherlanddw/cylc-run --status=succeeded --task=ext_check_globalfields_ukmet --point=20231031T00
no such table: suite_params

[sutherlanddw@w-ec-admin01 ~]$ cylc --version
7.8.11-30-gf9537
[sutherlanddw@w-ec-admin01 ~]$ cylc suite-state download/run1 --run-dir=/home/sutherlanddw/cylc-run --status=succeeded --task=ext_check_globalfields_ukmet --point=20231031T00
polling for 'succeeded': satisfied

[hjoliver]

@MetRonnie - now it's done!!

[MetRonnie]

Tested