Syntax Files #2752

oliver-sanders · 2018-08-14T09:25:37Z

Provide lexers for the Cylc suite definition syntax in different editors / systems.

Currently supported:

vi - etc/syntax/cylc.vim
emacs - etc/syntax/cylc-mode.el
gedit - etc/syntax/cylc.lang
kate - etc/syntax/cylc.xml
pygments - cylc-sphinx-extensions

Planned support:

A few major editors support the TextMate grammar format:

TextMate (TextMate)
Atom (TextMate or Tree-sitter)
Visual Code Studio (TextMate)
Ace (TextMate)

Dropped support:

google code prettify

Feature support table:

🆗 - Full Support
⛔ - Not Supported
⛅ - Partial Support

Feature	Example	`vi`	`emacs`	`gedit`	`kate`	`pygments`
1. Basic INI/Cylc Syntax
1.1 Sections / Settings	`[a]b=c`	⛅ [3]	⛅ [5, 6]	⛅ [5]	⛅ [5]	🆗
1.2 Comments	`a=c # c`	🆗	🆗	🆗	🆗	🆗
1.3 Multiline Str	`""" ...`	🆗	🆗	🆗	🆗	🆗
1.4 Include Files	`%include`	⛅ [1]	⛔	⛅ [8]	⛔	🆗
2. Exotic Cylc Syntax
2.1 Graph tasks + deps	`foo`, `=>`	⛔ [14]	🆗 [12]	⛅ [9]	🆗	🆗
2.2 Graph Comments	`foo # Foo`	🆗	🆗	🆗	🆗	🆗
2.3 Graph operators	`&`, `()`, `!`	⛔	⛔	🆗	⛅	🆗
2.4 Inter-Suite	`<a.b::c>`	⛔	⛔	⛔	⛔	🆗
2.5 xtriggers	`@wall_clock`	⛔	⛔	⛔	⛔	🆗
2.6 Parameter	`<x>`	⛔	⛔	⛔	⛔	🆗
2.7 Parameter	`<x-1>`	⛔	⛔	🆗	⛔	🆗
2.8 Parameter	`<x=1>`	⛔	⛔	🆗	⛔	🆗
2.9 Qualifier	`a:fail`	⛅ [2]	⛔	⛔	⛔	🆗
2.10 Inter-cycle	`a[-P1D]`	🆗	🆗	⛔	⛔	⛅ [13]
3. Templating
3.1 Shebang	`#!...`	🆗	⛔	🆗	🆗	🆗
3.2 Jinja2	`{%`, `{{`, `{#`	🆗	🆗 ~~[7]~~	⛅ [11]	🆗	🆗
3.3 Jinja2 Multiline	`{% \n %}`	🆗	🆗	🆗	⛔	🆗
3.4 empy (basic)	`@X`, `@(`, `@[`	⛅ [4]	⛔	⛔	⛔	🆗
3.5 empy (full)	`@?`, `@!`, `@...`	⛔	⛔	⛔	⛔	⛔

[1] Can't handle quoted file names.
[2] Problem with custom qualifiers containing underscores
[3] Equals characters in setting values breaks highlighting
[4] Parenthesis in expressions can cause issues
[5] Comments after sections break highlighting
[6] => symbols are highlighted outside of graph strings
~~[7] Cylc comments following Jinja2 blocks don't display as comments~~[Second observer: I don't see this issue?]
[8] Unquoted files aren't highlighted
[9] foo=>bar displays incorrectly
[10] Suicide triggers not highlighted
[11] Jinja2 comments not displayed as comments
[12] => highlighted, & tasks appear in the same mode as e.g. standard values under sections but
that seems appropriate?
[13] Arbitrary whitespace not supported in arthymatic (e.g. [^+P1D] is fine but [^ + P1D] is not).
[14] => acknowledged in triple-, but not single-, quoted graph strings.

Please update the table as syntax capabilities change.

2018-10-05 suite.rc syntax lexers: improve emacs support #2784
- emacs
  - Support multiline Jinja2 statements
  - Fix scoping issues
2019-01-07 pygments syntax: improve cylc lexer metomi/rose#2274
- pygments
  - Support inter-suite triggers
  - Support include files
  - Support xtriggers
  - Support empy
  - Support arthritic in inter-cycle triggers
  - Fix line continuation chars in headings

The text was updated successfully, but these errors were encountered:

sadielbartholomew · 2018-08-14T09:52:26Z

I volunteer myself as emacs rep! In the interests of achieving equality between the two most common editors, I am adding addressing Jinja2 Multiline (first & foremost) & EmPy to my To Do list.

(Oh & +1 for creative emoji use.)

hjoliver · 2018-08-14T10:02:53Z

@sadielbartholomew - I was an emacs user once, and then ...

hjoliver · 2018-08-14T10:04:47Z

(apologies 🐑-ish 😁)

sadielbartholomew · 2018-08-14T10:13:07Z

That made me 'laugh out loud', though I don't think I will be having such a revelatory moment (with vim certainly... maybe with Atom)!

hjoliver · 2018-08-14T10:14:14Z

To be fair it did take me a little while to see the light!

oliver-sanders · 2018-08-14T10:21:37Z

IDEs aside the top text editors from the StackOverflow survey:

sadielbartholomew · 2018-08-14T11:02:55Z

...okay so I was completely wrong about emacs & vim being the two dominant editors (I got this impression but admittedly never thought to investigate)! Those are interesting stats indeed.

hjoliver · 2018-08-14T20:41:25Z

There's also MS Visual Studio Code, a standalone code editor built on electron. It's pretty slick and has a pretty comprehensive vim emulation mode. I tried it a while ago, but I felt a bit "dirty", and I still like the lightness of vim.

https://en.wikipedia.org/wiki/Visual_Studio_Code

matthewrmshin · 2018-08-15T07:48:04Z

I have also tried VSCode recently (on my home Windows box and an Ubuntu VM). It is pretty good. Not sure if I can actually use it for real productivity work or not.

oliver-sanders · 2018-08-15T10:07:22Z

Added to the list, I thought it was an IDE (been a while since I used visual studio).

TomekTrzeciak · 2018-08-15T12:54:13Z

I've come across EmPy highlighting for Atom, maybe it will be useful:
https://github.com/dirk-thomas/language-empy

sadielbartholomew · 2018-11-01T14:32:05Z

(Updated main table in leading comment to reflect emacs mode update from merge of #2784.)

wxtim · 2020-02-11T20:24:57Z

n.b. I have started playing with with textMate for VSCode, and have got the basic bits working.

edmundhenley · 2020-03-07T10:28:52Z

Tim, as (now!) a VSCode user, who's just been hunting around a bit re options for rose/cylc, give me a shout if extra (noob-level) support on looking into textMate/being pilot user/reviewing docs/other helpful!
(have seen Oliver's recommendation elsewhere that one can ~use ini syntax rules for rose).

MetRonnie · 2020-03-26T15:07:51Z

Just to announce I've picked up on Tim's TextMate grammar for VSCode, should be ready to go in a few days time. As discussed in the last couple of Metomi video meetings, we'll need to figure out how we're going to host it, GitHub-repo-wise, considering it'll also be ported over to Atom

MetRonnie · 2020-04-08T19:06:40Z

The VSCode/TextMate grammar is in an almost deliverable state.

Feature	Example	`VSCode/TextMate`
1. Basic INI/Cylc Syntax
1.1 Sections / Settings	`[a]b=c`	🆗
1.2 Comments	`a=c # c`	🆗
1.3 Multiline Str	`""" ...`	🆗
1.4 Include Files	`%include`	🆗
2. Exotic Cylc Syntax
2.1 Graph tasks + deps	`foo`, `=>`	🆗
2.2 Graph Comments	`foo # Foo`	🆗
2.3 Graph operators	`&`, `()`, `!`	🆗
2.4 Inter-Suite	`<a.b::c>`	🆗
2.5 xtriggers	`@wall_clock`	🆗
2.6 Parameter	`<x>`	🆗
2.7 Parameter	`<x-1>`	🆗
2.8 Parameter	`<x=1>`	🆗
2.9 Qualifier	`a:fail`	🆗
2.10 Inter-cycle	`a[-P1D]`	🆗
3. Templating
3.1 Shebang	`#!...`	⛔
3.2 Jinja2	`{%`, `{{`, `{#`	🆗 [1,2]
3.3 Jinja2 Multiline	`{% \n %}`	🆗 [1,2]
3.4 empy (basic)	`@X`, `@(`, `@[`	⛔
3.5 empy (full)	`@?`, `@!`, `@...`	⛔

4. Extra
4.1 Cylc8 graph Syntax	`[[graph]]R1=...`	🆗
4.2 Shell script	`script = echo "foo"`	⛔
5. ISO 8601
5.1 Intervals	`P1Y1M1DT1H1M1S`	🆗
5.2 Datetime long	`2000-01-01T00:00:00-11:30`	🆗
5.2 Datetime short	`20051201T0215+0530`	🆗
5.3 Illegal intervals	`P1D1Y`, `P1H`	⛅
5.4 Illegal Datetime	`2000-01-01T0000`	⛅

[1] Enabled even if #!Jinja2 shebang not present
[2] ~~Jinja2 content all uniform (no further highlighting within Jinja2 blocks)~~ Can be used with a 3rd-party Jinja2 extension to provide Jinja2 highlighting inside cylc files

I've done it using a JavaScript file that can get 'compiled' to the JSON grammar file (used by VSCode and Atom). This makes it easier to reuse things when you have several similar rules.

The question is, how are we going to host it? I think we'll need separate repos for the VSCode and Atom versions of the grammar. How about an independent main repo for just the core TextMate grammar files (js, JSON and build scripts)? The VSCode and Atom repos could be modified clones and get merged into from the main repo, for new releases?

In any case, we could create the VSCode repo for now - should I create an empty repo then do a PR with all the commits so far?

oliver-sanders · 2020-04-08T21:42:59Z

The grammar ( and entire vscode plugin) are static so this isn’t really the domain of package managers like npm.

I think this might just be the kind of problem git Submodules we’re created to solve.

We would - Create a repo with the text mate extension in and a bunch of other repos for vscode, atom, heck maybe even textmate itself which use the same git Submodule.

Thoughts?

MetRonnie · 2020-04-09T09:24:43Z

I hadn't heard of submodules before but that looks promising

wxtim · 2020-04-14T10:09:01Z

I had a go at opening about 6 suites randomly picked from <site users path>/*/cylc-run/*/suite.rc when and they all looked Good. Tried it with one (I'll tell you which on a private channel) which I picked out as likely to push the envelope, and there are some crispy bits of jinja2 around the edges, but by-and-large it was ok.

hjoliver · 2020-04-15T22:38:16Z

Note our syntax support is far from perfect for any editor (I still use Vim mostly) so I woudn't worry too much about "the crispy bits" (nicely put @wxtim 😁 )

oliver-sanders · 2020-04-16T08:25:22Z

Agreed, just document the deficiencies in the table above, this is something we may get people in the Cylc community working on.

MetRonnie · 2020-05-04T15:56:52Z

(Posting here in addition to Discourse)

The extension for VSCode has been released: Visual Studio Marketplace; GitHub repo

Plus TextMate bundle for TextMate, PyCharm, WebStorm & Sublime Text 3: GitHub repo

Update: the package for Atom has been released: https://atom.io/packages/language-cylc

oliver-sanders added this to the later milestone Aug 14, 2018

oliver-sanders mentioned this issue Aug 14, 2018

EmPy as alternative to Jinja2 #2734

Merged

2 tasks

sadielbartholomew self-assigned this Aug 14, 2018

sadielbartholomew mentioned this issue Oct 5, 2018

suite.rc syntax lexers: improve emacs support #2784

Merged

sadielbartholomew removed their assignment Nov 2, 2018

This was referenced Jan 4, 2019

Docs (User + Design Guides) conversion to Sphinx #2910

Merged

pygments syntax: improve cylc lexer metomi/rose#2274

Merged

oliver-sanders assigned oliver-sanders and unassigned oliver-sanders Jan 7, 2019

This was referenced Jan 8, 2019

Testing for syntax files (lexers) #2915

Open

syntax: add reference syntax files for testing lexers #2916

Merged

matthewrmshin modified the milestones: later, cylc-9 Aug 28, 2019

oliver-sanders assigned MetRonnie Mar 26, 2020

This comment has been minimized.

Sign in to view

MetRonnie removed their assignment Aug 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Syntax Files #2752

Syntax Files #2752

oliver-sanders commented Aug 14, 2018 •

edited

Loading

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

hjoliver commented Aug 14, 2018

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

oliver-sanders commented Aug 14, 2018 •

edited by MetRonnie

Loading

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

matthewrmshin commented Aug 15, 2018

oliver-sanders commented Aug 15, 2018

TomekTrzeciak commented Aug 15, 2018

sadielbartholomew commented Nov 1, 2018

wxtim commented Feb 11, 2020

edmundhenley commented Mar 7, 2020 •

edited

Loading

MetRonnie commented Mar 26, 2020 •

edited

Loading

MetRonnie commented Apr 8, 2020 •

edited

Loading

oliver-sanders commented Apr 8, 2020 •

edited

Loading

MetRonnie commented Apr 9, 2020

This comment has been minimized.

wxtim commented Apr 14, 2020

hjoliver commented Apr 15, 2020

oliver-sanders commented Apr 16, 2020

MetRonnie commented May 4, 2020 •

edited

Loading

Syntax Files #2752

Syntax Files #2752

Comments

oliver-sanders commented Aug 14, 2018 • edited Loading

Provide lexers for the Cylc suite definition syntax in different editors / systems.

Currently supported:

Planned support:

Dropped support:

Feature support table:

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

hjoliver commented Aug 14, 2018

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

oliver-sanders commented Aug 14, 2018 • edited by MetRonnie Loading

sadielbartholomew commented Aug 14, 2018

hjoliver commented Aug 14, 2018

matthewrmshin commented Aug 15, 2018

oliver-sanders commented Aug 15, 2018

TomekTrzeciak commented Aug 15, 2018

sadielbartholomew commented Nov 1, 2018

wxtim commented Feb 11, 2020

edmundhenley commented Mar 7, 2020 • edited Loading

MetRonnie commented Mar 26, 2020 • edited Loading

MetRonnie commented Apr 8, 2020 • edited Loading

oliver-sanders commented Apr 8, 2020 • edited Loading

MetRonnie commented Apr 9, 2020

This comment has been minimized.

wxtim commented Apr 14, 2020

hjoliver commented Apr 15, 2020

oliver-sanders commented Apr 16, 2020

MetRonnie commented May 4, 2020 • edited Loading

oliver-sanders commented Aug 14, 2018 •

edited

Loading

oliver-sanders commented Aug 14, 2018 •

edited by MetRonnie

Loading

edmundhenley commented Mar 7, 2020 •

edited

Loading

MetRonnie commented Mar 26, 2020 •

edited

Loading

MetRonnie commented Apr 8, 2020 •

edited

Loading

oliver-sanders commented Apr 8, 2020 •

edited

Loading

MetRonnie commented May 4, 2020 •

edited

Loading