comment-preserving parser/generator?

[tve]

Has anyone written a comment-preserving parser/generator? The reason I'm asking is that I like config files, but I also like to be able to machine-edit them, for example through a web interface. Using the plain parser/generator one looses all comments in the config file.

[mojombo]

I'm not aware of any, but that would be a great addition to a TOML parser.

[mojombo]

I don't think you're going to get an answer here. Have you tried asking on Twitter or browsing through the list of parsers? Gonna close for now.

[kawing-chiu]

I think this feature should really go into the toml spec...

[TheElectronWill]

@tve @kawing-chiu If it helps, I've written one in java that preserve most comments (not 100%, e.g. comments in arrays are not preserved for now). It's part of the Night-Config library.

[tshepang]

tomllib (written in Rust) preserves comments I believe

[shadow-light]

Python tomlkit preserves comments. In fact that is why I was going to change all of my projects to TOML! Since learning it isn't actually in the spec I'm not so sure now. Putting this in the spec would give a huge advantage in my opinion over other config languages.

[ChristianSi]

@shadow-light Why should something like that be in the spec? Parsers/writers can implement things like preserving comments and order of elements, but most TOML apps will only read or write TOML files, not both, hence preserving syntactic sugar is irrelevant for them. If Python tomlkit preserves such details, that might be nice for some use cases (say a built-in editor for your TOML config), so by all means use it if it fits your needs.

Just remember that TOML tables are unordered so you cannot require your users to write keys in a specific order; and, since comments are ignored, you cannot rely on the absence or presence of specific comments. I think that's a good thing.

[shadow-light]

Because comments are highly valuable sources of information to remember why certain values have been set. When you've written a lot of comments in a config file, it is very annoying to have them all wiped out because one of the values needed to be updated by a CLI for example.

It is sufficient for my use cases at least that the Python implementation does this. But putting it in the spec would guarantee it stays that way, and the feature doesn't get removed, or one can switch to another language implementation and not worry.

[bradleypeabody]

most TOML apps will only read or write TOML files, not both

While I understand why this comment was made, I don't think it's a safe assumption at all. It is also a sort of natural progression to start with a simple hand-edited configuration file and then want to add a UI or other automation to it later. I've literally encountered this problem on nearly every significantly-sized project I've worked on.

[federico-terzi-anima]

For people looking for a solution to this in 2024, in Rust we have the toml_edit crate: https://crates.io/crates/toml_edit