Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-85308: argparse: Use filesystem encoding for arguments file #93277

Merged
merged 3 commits into from
Jun 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Doc/library/argparse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -562,7 +562,7 @@ at the command line. If the ``fromfile_prefix_chars=`` argument is given to the
specified characters will be treated as files, and will be replaced by the
arguments they contain. For example::

>>> with open('args.txt', 'w') as fp:
>>> with open('args.txt', 'w', encoding=sys.getfilesystemencoding()) as fp:
... fp.write('-f\nbar')
>>> parser = argparse.ArgumentParser(fromfile_prefix_chars='@')
>>> parser.add_argument('-f')
Expand All @@ -575,9 +575,18 @@ were in the same place as the original file referencing argument on the command
line. So in the example above, the expression ``['-f', 'foo', '@args.txt']``
is considered equivalent to the expression ``['-f', 'foo', '-f', 'bar']``.

:class:`ArgumentParser` uses :term:`filesystem encoding and error handler`
to read the file containing arguments.

The ``fromfile_prefix_chars=`` argument defaults to ``None``, meaning that
arguments will never be treated as file references.

.. versionchanged:: 3.12
:class:`ArgumentParser` changed encoding and errors to read arguments files
from default (e.g. :func:`locale.getpreferredencoding(False)` and
``"strict"``) to :term:`filesystem encoding and error handler`.
Arguments file should be encoded in UTF-8 instead of ANSI Codepage on Windows.


argument_default
^^^^^^^^^^^^^^^^
Expand Down
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,12 @@ Changes in the Python API
select from a larger range than ``randrange(10**25)``.
(Originally suggested by Serhiy Storchaka gh-86388.)

* :class:`argparse.ArgumentParser` changed encoding and error handler
for reading arguments from file (e.g. ``fromfile_prefix_chars`` option)
from default text encoding (e.g. :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>`)
to :term:`filesystem encoding and error handler`.
Argument files should be encoded in UTF-8 instead of ANSI Codepage on Windows.


Build Changes
=============
Expand Down
4 changes: 3 additions & 1 deletion Lib/argparse.py
Original file line number Diff line number Diff line change
Expand Up @@ -2161,7 +2161,9 @@ def _read_args_from_files(self, arg_strings):
# replace arguments referencing files with the file content
else:
try:
with open(arg_string[1:]) as args_file:
with open(arg_string[1:],
encoding=_sys.getfilesystemencoding(),
errors=_sys.getfilesystemencodeerrors()) as args_file:
arg_strings = []
for arg_line in args_file.read().splitlines():
for arg in self.convert_arg_line_to_args(arg_line):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Changed :class:`argparse.ArgumentParser` to use :term:`filesystem encoding
and error handler` instead of default text encoding to read arguments from
file (e.g. ``fromfile_prefix_chars`` option). This change affects Windows;
argument file should be encoded with UTF-8 instead of ANSI Codepage.