Fix generate_system_pyproject newline issues

[brooke-ec]

Addressed Issues

Additional Carriage Returns

tomlkit persists the original line separators for the preserved tables, as a result these lines will be separated by \r\n on WIndows. Since the newline kwarg is omitted from the self.system_pyproject.write_text^[1] method call, all occurrences of \n are replaced by os.linesep^[2] before writing. This causes the resultant line separators for the preserved lines to be \r\r\n on Windows.

These additional carriage returns become an issue when the file is parsed by tomli, which replaces only \r\n with \n,^[3] leaving the extra return. When encountering this, the following exception will be raised: TOMLDecodeError: Expected newline or end of document after a statement (at line 12, column 44).

This issue causes the following commands to prematurely exit on Windows when any plugins are installed: self lock, self show, self update and self show plugins.

Increasing Lines at End of File

tomlkit also persists the number of blank lines after an entry, as a result when preserved tables are present, the newline added by Factory.create_pyproject_from_package^[4] is unnecessary and will cause a new line to be added to the file every time the function is called.

Changes To Fix The Issues

1. Use TOMLFile.write for saving the file to let tomlkit handle the translation of newlines.
2. Remove line that adds the trailing newline in Factory.create_pyproject_from_package.

Pull Request Check List

Resolves #6457

• Added tests for changed code.
• Updated documentation for changed code.

[dimbleby]

Something like

-        self.system_pyproject.write_text(content.as_string(), encoding="utf-8")
+        pyproject = PyProjectTOML(self.system_pyproject)
+        pyproject.file.write(content)

should be cleaner, let tomlkit worry about the line endings. (Which it does, see https://github.com/sdispater/tomlkit/blob/18a527758bfa3f8fe65e2f45f61df42f2f5af0c1/tomlkit/toml_file.py#L47)

per the pipeline, write_text() doesn't get the newline parameter until python 3.10.

[brooke-ec]

Thanks for the help! That is definitely a cleaner solution. It's just a case of finding a nice solution to prevent those newlines from being added with this new method.

[dimbleby]

well, one thing at a time. Although the extra lines and the faulty carriage returns are in the same general area they're really quite different; and one oughtn't cause us to make a worse fix to the other.

re the unwanted newlines: do we ever want that newline? perhaps just delete the line of code that adds it altogether. Or, make it caller responsibility to add it if they want it.

[brooke-ec]

The newline appears to be added solely for code styling, so I'd say it is probably alright to remove. Aside from here, the Factory.create_pyproject_from_package method is only used in test_factory.py and test_remove_plugins.py, neither of which look like they will be effected by the removal of this newline.

[radoering]

I'd propose to rename the test file to test_self_command.py. Usually, we use the same name as the module we are testing.

[dimbleby]

fixes #6457, presumably

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.