Added the ability to dump the token stream or ast in bin. #278
Conversation
The dump functionality works both for files and REPL. With --dump-tokens or -t for short it dumps the token stream to stdout and --dump-ast or -a for short to dump the ast to stdout. The dumping of tokens and ast is mutually exclusive. and when dumping it wont run the code.
|
Seems nice, maybe it would be interesting to print tokes/ast in a given format (JSON for example) in order for other tools to analyze it? |
|
Hmmm. That seems like a really nice idea. Thanks 👍 |
- Changed the --dump-tokens and --dump-ast to be an optional argument that optionally takes a value of format type ([--opt=[val]]). - The default format for --dump-tokens and --dump-ast is Debug format which calls std::fmt::Debug. - Added Json and JsonMinified format for both dumps, use serde_json internally. - It is easy to support other format types, such as Toml with toml-rs for example.
|
With the last commit 84d4105, I changed --dump-tokens and --dump-ast to optionally take a format type, by default it is Debug format which internally uses the std::fmt::Debug trait. But you can change it by providing a format type like: Debug, Json, JsonMinified. The input value is case insensitive. Example: cargo run -- test.js --dump-tokens #=Debug or cargo run -- test.js --dump-tokens=Json # pretty printed json.or cargo run -- test.js --dump-tokens=JsonMinified # minified json.Its the same with --dump-ast. |
|
This looks great @HalidOdat i will take a proper look soon but no issues so far |
|
It looks great, yes! Maybe the And I would say that if we want output in JSON, it should be minified by default, with the option to make it pretty if desired. |
|
Thanks for the input. I will see what I can do. |
- Serde serialization and deserialization can be switched on by using the feature flag "serde-ast".
- Now Json dumping format prints the data in minefied JSON form by default. - Removed JsonMinified. - Added JsonPretty as a way to dump the data in pretty printed JSON format.
|
With this commit 8a34176, I made And with this commit 96c74ae, I changed Json format to be minified by default. Removed the JsonMineified format and added JsonPretty, which as the name implies it dumps with pretty print JSON format. |
|
Looks perfect to me, great job! |
|
Thanks! |
|
Maybe we can get this in before the parser change then rebase the parser branch on top of this? |
|
I think so, too. |
The dump functionality works both for files and REPL.
With --dump-tokens or -t for short it dumps the token stream to stdout and --dump-ast or -a for short to dump the ast to stdout.
The dumping of tokens and ast is mutually exclusive and when dumping it wont run the code.