Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: New list syntax with enforced ,-operator #100

Merged
merged 42 commits into from
Dec 20, 2023

Conversation

phorward
Copy link
Member

@phorward phorward commented Jan 9, 2023

This pull request initially was started to evolve a new syntax for lists using the [...] brackets, like in Python. But in Tokays sequence/parsing behavior, it worked out that this syntax won't be useful, because it becomes just another type of sequence. Therefore, the previously optional comma , was re-defined, and now defines a list within this pull request.

Therefore, the following examples and rules apply:

d = ()  # this is now a dict
l = ,  # this is now a list
l = (,)  # is equal to above

l = 1,  # list with one item
l = (1,)  # is equal to above
l = 1, 2, 3  # list with 3 items
l = 1, (2, 3, 4)  # list with 2 items, second a list with 3 items

# mixed syntax not allowed
d = (1, b => 2, 3)  # previously accepted; now a syntax error.
d = (1 b => 2 3)  # clarified: it is a dict equal to (0 => 1 b => 2 2 => 3)
d = (1 b => (2, 3))  # clarified: it is a dict equal to (0 => 1 b => (2, 3))
l = (1, (b => 2), 3)  # clarified: it is a list, with a dict in l[1]

Initial comment:

This pull requests drafts and implements an explicit list syntax for Tokay.
Similar to Python and Rust, it uses [...] to specify explicit lists.

This eliminates the following problems Tokay has so far

This PR is a draft and under development.


* `(...)` either produces dicts or lists, `()` is the explicit dict

This isn't useful. Currently, (1 2 3) generates repr [1, 2, 3].

Short roadmap on this topic:

  1. (...)-syntax is called a dict-bound sequence and should always produce dicts. (1 2 3) should reproduce repr (1, 2, 3).
  2. [...]-syntax is called a list-bound sequence and always produces lists. It should allow for |-operator as well, equally to dict-bound syntax, to specify inline sequences
  3. The AST traversal, respectively object traits, should allow iteration over either a list or a dict's values in order.
  4. The pos/kle repeat-constructs currently either produce a single dict or a list of dicts. This is should be changed to always product a list of 1..n items, or in case of kle, void, if no match is found.

It should also be considered that this PR is being continued when #10 (#105) is generally implemented, so that several constructs become easier to define.

- sequence mode is the previous behavior
- void-values with an alias become null values
- non-sequence mode always results in a dict (probably empty)
This is unfinished and intermediate, as the syntax is still not clear between inline sequences and dicts.

Further development depends on iterators.
@phorward phorward added feature New feature or request discussion Further discussion on this topic required syntax labels Jan 9, 2023
@phorward
Copy link
Member Author

phorward commented Jan 10, 2023

(Moved roadmap to top)

@phorward phorward removed the tokay label Oct 8, 2023
@phorward phorward removed discussion Further discussion on this topic required pending labels Oct 22, 2023
Still unfinished. Trying several ways.
This branch starts over with another syntax for lists, where the comma (,) becomes an operator with a meaning.
- Clean-up and fixing grammar
- Updating and investigating tests
- Updating prelude
@phorward phorward changed the title Implementing explicit list syntax [...] Implementing explicit list syntax using , Nov 10, 2023
- InlineAssignment separates the assigned value on the stack
- Enhanced dict.tok dests, fixes test suite in this branch for now
@phorward phorward changed the title Implementing explicit list syntax using , Implementing list syntax using , as operator Nov 18, 2023
@phorward phorward marked this pull request as ready for review December 7, 2023 00:10
@phorward phorward changed the title Implementing list syntax using , as operator feat: Implementing list syntax using , as operator Dec 20, 2023
@phorward phorward changed the title feat: Implementing list syntax using , as operator feat: New list syntax with enforced ,-operator Dec 20, 2023
@phorward phorward merged commit a08e12f into tokay-lang:main Dec 20, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request syntax
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement dict as IndexMap<RefValue, RefValue> when key is not mutable
1 participant