Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly handle newlines after/before comments #4895

Merged
merged 3 commits into from
Jun 7, 2023

Conversation

MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Jun 6, 2023

Summary

This issue fixes the removal of empty lines between a leading comment and the previous statement:

a  = 20

# leading comment
b = 10

Ruff removed the empty line between a and b because:

  • The leading comments formatting does not preserve leading newlines (to avoid adding new lines at the top of a body)
  • The JoinNodesBuilder counted the lines before b, which is 1 -> Doesn't insert a new line

This is fixed by changing the JoinNodesBuilder to count the lines instead after the last node. This correctly gives 1, and the # leading comment will insert the empty lines between any other leading comment or the node.

Test Plan

I added a new test for empty lines.

@MichaReiser
Copy link
Member Author

MichaReiser commented Jun 6, 2023

@MichaReiser MichaReiser added internal An internal refactor or improvement formatter Related to the formatter labels Jun 6, 2023
@MichaReiser MichaReiser marked this pull request as ready for review June 6, 2023 07:45
@MichaReiser MichaReiser requested a review from konstin June 6, 2023 07:45
@MichaReiser MichaReiser force-pushed the format-binary-expression branch from 5dde064 to 5645338 Compare June 6, 2023 07:52
@github-actions
Copy link
Contributor

github-actions bot commented Jun 6, 2023

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.01      6.2±0.01ms     6.5 MB/sec    1.00      6.2±0.02ms     6.6 MB/sec
formatter/numpy/ctypeslib.py               1.00   1265.5±1.59µs    13.2 MB/sec    1.00   1259.7±1.93µs    13.2 MB/sec
formatter/numpy/globals.py                 1.01    145.5±0.27µs    20.3 MB/sec    1.00    144.5±1.73µs    20.4 MB/sec
formatter/pydantic/types.py                1.00      2.7±0.01ms     9.4 MB/sec    1.00      2.7±0.01ms     9.4 MB/sec
linter/all-rules/large/dataset.py          1.00     14.8±0.07ms     2.7 MB/sec    1.00     14.8±0.08ms     2.7 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.6±0.00ms     4.7 MB/sec    1.00      3.6±0.00ms     4.7 MB/sec
linter/all-rules/numpy/globals.py          1.00    362.6±0.86µs     8.1 MB/sec    1.00    362.2±1.17µs     8.1 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.1±0.01ms     4.1 MB/sec    1.00      6.2±0.02ms     4.1 MB/sec
linter/default-rules/large/dataset.py      1.00      7.4±0.01ms     5.5 MB/sec    1.00      7.4±0.02ms     5.5 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1544.3±2.64µs    10.8 MB/sec    1.00   1542.8±4.09µs    10.8 MB/sec
linter/default-rules/numpy/globals.py      1.00    164.2±0.47µs    18.0 MB/sec    1.00    164.4±0.48µs    17.9 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.3±0.00ms     7.7 MB/sec    1.00      3.3±0.00ms     7.7 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.04      7.2±0.07ms     5.7 MB/sec    1.00      6.9±0.06ms     5.9 MB/sec
formatter/numpy/ctypeslib.py               1.05  1445.7±24.58µs    11.5 MB/sec    1.00  1377.2±17.74µs    12.1 MB/sec
formatter/numpy/globals.py                 1.03    161.9±3.70µs    18.2 MB/sec    1.00    156.7±3.96µs    18.8 MB/sec
formatter/pydantic/types.py                1.04      3.2±0.10ms     8.1 MB/sec    1.00      3.0±0.04ms     8.4 MB/sec
linter/all-rules/large/dataset.py          1.00     16.6±0.17ms     2.4 MB/sec    1.00     16.7±0.14ms     2.4 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      4.2±0.04ms     4.0 MB/sec    1.01      4.2±0.05ms     4.0 MB/sec
linter/all-rules/numpy/globals.py          1.00    488.8±6.90µs     6.0 MB/sec    1.01    493.3±8.37µs     6.0 MB/sec
linter/all-rules/pydantic/types.py         1.00      7.0±0.08ms     3.6 MB/sec    1.01      7.1±0.07ms     3.6 MB/sec
linter/default-rules/large/dataset.py      1.00      8.2±0.06ms     5.0 MB/sec    1.02      8.4±0.06ms     4.9 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00  1737.0±29.96µs     9.6 MB/sec    1.01  1762.5±19.00µs     9.4 MB/sec
linter/default-rules/numpy/globals.py      1.00    196.1±4.39µs    15.0 MB/sec    1.01    198.9±3.03µs    14.8 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.7±0.04ms     6.9 MB/sec    1.02      3.8±0.05ms     6.7 MB/sec

Base automatically changed from format-binary-expression to main June 6, 2023 08:34
@MichaReiser MichaReiser force-pushed the comments-newline-handling branch from 8e5706c to b2ec055 Compare June 6, 2023 18:22
// Skip the comment
let newline_offset = iter
.as_str()
.find(['\n', '\r'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this handle CR LF correctly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Because we are only interested in finding the offset before the newline. It then doesn't matter whether it is \r, \n or \r\n

@MichaReiser
Copy link
Member Author

@MichaReiser started a stack merge that includes this pull request via Graphite.

@MichaReiser MichaReiser merged commit 6ab3fc6 into main Jun 7, 2023
@MichaReiser MichaReiser deleted the comments-newline-handling branch June 7, 2023 12:49
@MichaReiser
Copy link
Member Author

@MichaReiser merged this pull request with Graphite.

konstin pushed a commit that referenced this pull request Jun 13, 2023
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This issue fixes the removal of empty lines between a leading comment and the previous statement:

```python
a  = 20

# leading comment
b = 10
```

Ruff removed the empty line between `a` and `b` because:
* The leading comments formatting does not preserve leading newlines (to avoid adding new lines at the top of a body)
* The `JoinNodesBuilder` counted the lines before `b`, which is 1 -> Doesn't insert a new line

This is fixed by changing the `JoinNodesBuilder` to count the lines instead *after* the last node. This correctly gives 1, and the `# leading comment` will insert the empty lines between any other leading comment or the node.



## Test Plan

I added a new test for empty lines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
formatter Related to the formatter internal An internal refactor or improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants