Parsing of generators allows skipping a comma

[savq]

JuliaSyntax allows skipping a separator after a generator. In Julia 1.10 you have:

:(x for x in xs a, b) == :((x for x in xs), a, b)

Even more strangely, depending on the separator that comes after the expression may be parsed as a block or as a tuple:

julia> parsestmt(SyntaxNode, "(x for x = xs a; b)")
line:col│ tree                                   │ file_name
   1:1  │[block-p]
   1:2  │  [generator]
   1:2  │    x
   1:7  │    [=]
   1:8  │      x
   1:12 │      xs
   1:15 │  a
   1:18 │  b


julia> parsestmt(SyntaxNode, "(x for x = xs a, b)")
line:col│ tree                                   │ file_name
   1:1  │[tuple-p]
   1:2  │  [generator]
   1:2  │    x
   1:7  │    [=]
   1:8  │      x
   1:12 │      xs
   1:15 │  a
   1:18 │  b

This error isn't present in the flisp parser. The error is present in the flisp parser, but only in certain contexts. In Julia 1.9, we have:

julia> :(x for x = xs a) # plain generator
ERROR: syntax: expected ")"
Stacktrace:
 [1] top-level scope
   @ none:1

julia> :(f(x for x = xs a)) # Generator inside call
:(f((x for x = in), a))

[c42f]

Ok. Yes yuck! JuliaSyntax shares the old flisp bug too

julia> parsestmt(SyntaxNode, "f(x for x = xs a)")
SyntaxNode:
[call]
  f                                      :: Identifier
  [generator]
    x                                    :: Identifier
    [iteration]
      [in]
        x                                :: Identifier
        xs                               :: Identifier
  a                                      :: Identifier'

[c42f]

We also have these bugs in the flisp parser:

julia> JuliaSyntax.fl_parse("(a, x for x in xs c)")
:((a, (x for x = xs), c))

julia> dump(JuliaSyntax.fl_parse("(a, x for x in xs c)"))
Expr
  head: Symbol tuple
  args: Array{Any}((3,))
    1: Symbol a
    2: Expr
      head: Symbol generator
      args: Array{Any}((2,))
        1: Symbol x
        2: Expr
          head: Symbol =
          args: Array{Any}((2,))
            1: Symbol x
            2: Symbol xs
    3: Symbol c

julia> JuliaSyntax.fl_parse("(a; x for x in xs c)")
:((a,; (x for x = xs), c))

julia> dump(JuliaSyntax.fl_parse("(a; x for x in xs c)"))
Expr
  head: Symbol tuple
  args: Array{Any}((2,))
    1: Expr
      head: Symbol parameters
      args: Array{Any}((2,))
        1: Expr
          head: Symbol generator
          args: Array{Any}((2,))
            1: Symbol x
            2: Expr
              head: Symbol =
              args: Array{Any}((2,))
                1: Symbol x
                2: Symbol xs
        2: Symbol c
    2: Symbol a

[c42f]

The flisp parser also allows the following form and Base uses it :-(

julia> JuliaSyntax.fl_parse("f(y for x = xs; a)")
:(f((y for x = xs); a))

But the following is disallowed

julia> JuliaSyntax.fl_parse("(y for x = xs; a)")
ERROR: ParseError("expected \")\"")
Stacktrace:
 [1] #fl_parse#129
   @ ~/.julia/dev/JuliaSyntax/src/hooks.jl:366 [inlined]
 [2] fl_parse
   @ ~/.julia/dev/JuliaSyntax/src/hooks.jl:362 [inlined]
 [3] fl_parse(str::String; raise::Bool, depwarn::Bool)
   @ JuliaSyntax ~/.julia/dev/JuliaSyntax/src/hooks.jl:351
 [4] fl_parse(str::String)
   @ JuliaSyntax ~/.julia/dev/JuliaSyntax/src/hooks.jl:350
 [5] top-level scope
   @ REPL[44]:1