Keep comments above a key-value pair as is

[webartifex]

I tried out your library and it is really helpful. Thank you for your contribution.

Question: Do you think it is worthwhile to add an option to preserve (multi-line) comments that precede a key-value pair?

Example (from a pyproject.toml in one of my projects):

[tool.poetry.dependencies]
click = "^7.1.2"
coverage = {extras = ["toml"], optional = true, version = "^5.1"}
pytest = {optional = true, version = "^5.4.3"}
pytest-cov = {optional = true, version = "^2.10.0"}
pytest-xdist = {optional = true, version = "^1.32.0"}
python = "^3.8"
tox = {optional = true, version = "^3.16.1"}

[tool.poetry.extras]
# line1
# line2
# line3
develop = "..."
testing = "..."

[tool.poetry.scripts]
version = "..."

[pappasam]

Sadly, I can't think of a way to generally support keeping all but leading comments. Most comments I've seen precede the concept they're commenting on. For example, here's a pattern I wish I could support:

# this section talks about the dependencies section
[dependencies]
# dep1 is super important
dep1 = "^0.1"
# dep2 may be less useful
dep2 = "^0.2"

It'd be nice to keep each comment above its corresponding section in the sorted output.

That said, given the way people write comments, there's no standard location for comments on their own line. There's nothing wrong with your comment style, and by supporting the pattern I've described here, I'd probably be annoying you.

Additionally, in terms of working with tomlkit, I have a hard time working with comments that are neither at the beginning of the document nor inline.

In short, this problem is hard to address, both technically and otherwise. Do you have any ideas to address either challenge?

[webartifex]

You raised an issue I haven't seen before. At first glance, one option could be to provide the two or three most common comments usages as a flag option.

In the case you describe I would suggest the following behavior:

• "Attach" the n lines of comments preceding any single line of TOML to this one line conceptually
• Sort the TOML lines according to the currently implemented ways
• "Paste" the n lines before the TOML line it was attached to.

Adapting your example:

 # this section talks about the dev-dependencies section
[dev-dependencies]
# dep3 is super important
# it really is important
dep3 = "^0.1"
# dep4 may be less useful
dep4 = "^0.2"

# this section talks about the dependencies section
[dependencies]
# dep2 is super important
dep2 = "^0.1"
# dep1 may be less useful
# it really is important
dep1 = "^0.2"

becomes

# this section talks about the dependencies section
[dependencies]
# dep1 may be less useful
# it really is important
dep1 = "^0.2"
# dep2 is super important
dep2 = "^0.1"

# this section talks about the dev-dependencies section
[dev-dependencies]
# dep3 is super important
# it really is important
dep3 = "^0.1"
# dep4 may be less useful
dep4 = "^0.2"

[pappasam]

True. But don't forget that not all users will have consistent usage within their document. Especially for large toml files, users may prefer leading comments in one part and trailing comments in another.

That said, I'm not opposed to adding multiple flags with proper documentation. As I said earlier, from a technical perspective, I don't have any ideas for how to implement this at this time. If you want to play around with tomlkit / https://github.com/pappasam/toml-sort/blob/master/toml_sort/tomlsort.py , by all means! Otherwise, I did spend a while trying to make comments more flexible a while ago, to no avail :(

[pappasam]

The best I could do was to preserve leading comments.

[webartifex]

I'll take a look it once I got time. I bet this is solvable somehow. Thanks for your efforts so far :)

[xpe]

I appreciate this thread; it is a very useful conversation.

@webartifex I found this issue because I had the same question. You wrote:

In the case you describe I would suggest the following behavior:

• "Attach" the n lines of comments preceding any single line of TOML to this one line conceptually
• Sort the TOML lines according to the currently implemented ways
• "Paste" the n lines before the TOML line it was attached to.

Hello @pappasam -- this tool is wonderful, so thank you! Last year, you wrote:

That said, I'm not opposed to adding multiple flags with proper documentation. As I said earlier, from a technical perspective, I don't have any ideas for how to implement this at this time. If you want to play around with tomlkit / https://github.com/pappasam/toml-sort/blob/master/toml_sort/tomlsort.py , by all means! Otherwise, I did spend a while trying to make comments more flexible a while ago, to no avail :(

In response, I have a few questions:

• Is the definition of correct behavior (for each flag) well-defined?
• If not, what makes it ill-defined? Could you share an example or two showing this?

[pappasam]

@xpe one silly, yet clear, example:

[c]
x = 3
[b]
x = 2
# the preceding section's value was 2, which differs from the following section whose value is 1
[a]
x = 1

It's unclear where to place this comment in a sorted result because it references relative lines before and after itself.

The general problem is that comments are contextual and do not always only reference the lines that follow. This makes them hard to sort, in a general sense.

[pappasam]

And like comments themselves, the potential cli flags / options / whatever you want to call them are not well defined

[mlschechter]

Just a question - your documentation explicitly states this tool preserves inline comments. Given what you've stated here, that seems impossible to achieve.

Could you please let me know if the documentation is correct, and if so, what it refers to?

Thanks!

[pappasam]

@mlschechter by "inline comment", I mean a comment at the end of a line containing non-comment toml.

  title = "The example"

# I am not inline
[[a-section.hello]]
ports = [ 8001, 8001, 8002 ] # I am inline
dob = 1979-05-27T07:32:00Z # So am I

[mlschechter]

Thanks for the clarification!