Inspiration

[thchha]

Hey 6cdh,

just wanted to let you know that I have a working R5RS and most of R7RS tree-sitter grammar.
Maybe you want to join efforts.
Let me know.

Thomas

[6cdh]

Thank you for the information.

Do you mean this project gitlab:thchha/tree-sitter-scheme?
I have looked at it and it appears that it still lacks some features.
My grammar, on the other hand, is quite complete and optimized for small parser.c output and speed.
It's also generally smaller and faster than other lisp grammars.
I think it would be better on improving and extending my existing grammar instead of
make a new one.
If you have any ideas for new features or improvements,
please feel free to open an issue and we can discuss them there.

Thank you again for reaching out.

[thchha]

Exactly this grammar.

I need to be able to judge about the kind of syntax-node. I wasn't able to archive this via a patch on your grammar so had to restart. Note that I abstracted the BNF-grammars of said RnRS, so it should be fairly complete.
Anyhow, once you have support for the kind of nodes I am interested, I will join yours, then.

For example, I need to be able to identify the following S-Exp as a call to define via tree-sitter queries (without relying on regex, searches for each and every node):

(define (main . args)
  (map pp args)
  (exit 0))

We should talk again before implementing a custom-parser.c each on their own, I think.
Keep up your work.
Thank you as well!

[6cdh]

I considered this problem when I started to implement it.

Note here I will use the term syntax for define, let, etc.
Here are some reasons that I didn't choose to recognize the builtin Scheme syntax in the parser:

• syntax can be redefined at runtime: (define define 1)
• people might write their own control flow: (while cond body ...)

Both them make the statically recognization of the syntax is impractical.
Instead, I decided to parse sexp directly into list just like the read function in Scheme.
In this way, people can't get output like variable_definition, module_import directly from parser,
but they can still write custom queries for their own needs if the builtin queries is not enough.

For define a function, I think this query is fine:

(list
  .
  (symbol) @keyword
  .
  (list
    .
    (symbol) @function
    .
    [((symbol) @dot
      (symbol) @arguments)?
     (symbol) @argument]*
  (#eq? @keyword "define")
  (#eq? @dot ".")))

[thchha]

My current use is rather domain specific and I am able to work around this.
But given the broad variances of application for a tree-sitter grammar you have made the correct decision. That's thoughtfully crafted.

I think the context of this conversation and your provided listing is a concise and valuable hint for new users to your grammar.
At least I didn't thought about making heavy use of tree-sitter predicates and would suggest a short reasoning on your README.

Also the bindings to tree-sitter for gambit-C I created do not use a RegEx engine yet, so making proper use of tree-sitter predicates is currently prohibited for me. But this isn't an issue of your grammar should be noted.

[6cdh]

Okay. I remembered that the query are supported by the tree-sitter library
and the matches and captures can be iterated by some API.
I'm not sure as I never used it.

I will link README to this issue, and your repo for the people who don't like
queries and still want syntax in the parser if you think it's fine.

[thchha]

Yes, tree-sitter offers such an API.
The actual implementation of the predicates is provided by the bindings to tree-sitter in use.
That's a very good design, imho, but it lacks consistency.

And one has to grasp the way the query patterns are iterated on each step of a query. So there's that.

Thank you for your input. If I need to implement a custom-parser.cc for any language construct, I will get in touch again, if that is okay. And I will start watching, so that I can jump ships, when I can suffice my current requirements 👍