pygments syntax: improve cylc lexer

[oliver-sanders]

Support additional features of the Cylc suite.rc language to permit use in the Cylc documentation (cylc/cylc-flow#2910).

Adds support for the following (see table in cylc/cylc-flow#2752):

• Inter-Suite
• Include files
• xtriggers
• empy

Fixes:

• Line continuation characters in headings

   [foo, \
   bar]
  

• Cycle points in inter-cycle offsets

[matthewrmshin]

Changes look good by eyeballs 👀 (+ 👓 in my case). Do we need some automated tests?

[oliver-sanders]

Automated testing for syntax files, hmmm, that might be a tricky one.

The simplest thing I can think of doing would be to come up with some devilishly convoluted examples to run through each editor in turn, then perform a visual diff on before and after screenshots. We need to look into this stuff anyway for future GUI development ...

[matthewrmshin]

If there is a way to test by giving the lexical analyser some suite.rc fragments and see what tokens come out on the other side, then we should try that. Otherwise, change looks OK.

[oliver-sanders]

If there is a way to test by giving the lexical analyser some suite.rc fragments and see what tokens come out on the other side

For pygments, yes.
For vim, possibly.
For gedit god knows.

See cylc/cylc-flow#2915 for a discussion on testing approaches for syntax files.

[oliver-sanders]

Have added standard reference files for testing lexers in cylc/cylc-flow#2916.

PS/ Spotted an edge case with empy (@[if []]).

[sadielbartholomew]

I have noticed one omission. Dependencies on specific cycle points e.g. graph = "red[2019] => blue" are supported as valid specification within square brackets in a graph string, where only intercycle offsets appear to be recognised by the lexer as-is.

[oliver-sanders]

Drat.

Turns out you can also do arithmetic (e.g. [^ + P1D]) which I didn't know. This is going to be a little trickier to support whilst maintaining error checking.

[sadielbartholomew]

The good news is the lexer is now so water-tight it picked up on several missing markers in some of the Cylc documentation code excerpts. It's nearly there 👏

[sadielbartholomew]

(Very sorry for the delay in re-reviewing, I didn't realise commented reviews took PRs off the GitHub 'review requests' listing.)

Works as described. Lack of associated tests justified by comment.

To note: dependencies on specific cycle points are recognised after merge of this PR, but cycle point arithmetic has not been included in the PR domain & should eventually be incorporated:

[oliver-sanders]

cycle point arithmetic has not been included

Arithmetic is fine but arbitrary whitespace is not supported.

Table updated accordingly

[sadielbartholomew]

Arithmetic is fine but arbitrary whitespace is not supported.

Oh, fair enough, I simply used the example you provided in your original comment, assuming it was valid. Good!