PEP 678: Update proposal with .add_note() and acknowledgements section

pep-0678.rst

@@ -17,9 +17,10 @@ Abstract
 Exception objects are typically initialized with a message that describes the
 error which has occurred.  Because further information may be available when
 the exception is caught and re-raised, or included in an ``ExceptionGroup``,
-this PEP proposes to add a ``.__note__`` attribute and update the builtin
-traceback formatting code to include it in the formatted traceback following
-the exception string.
+this PEP proposes to add ``BaseException.add_note(note, *, replace=False)``, a
+``.__notes__`` attribute holding a tuple of zero or more notes so added, and to
+update the builtin traceback formatting code to include notes in the formatted
+traceback following the exception string.
 
 This is particularly useful in relation to :pep:`654` ``ExceptionGroup``\ s,
 which make previous workarounds ineffective or confusing.  Use cases have been
@@ -41,15 +42,19 @@ example,
   timestamp, or other explanation with each of several errors - especially if
   re-raising them in an ``ExceptionGroup``.
 - programming environments for novices can provide more detailed descriptions
-  of various errors, and tips for resolving them (e.g. ``friendly-traceback``).
+  of various errors, and tips for resolving them.
 
 Existing approaches must pass this additional information around while keeping
 it in sync with the state of raised, and potentially caught or chained,
 exceptions. This is already error-prone, and made more difficult by :pep:`654`
 ``ExceptionGroup``\ s, so the time is right for a built-in solution.  We
-therefore propose to add a mutable field ``__note__`` to ``BaseException``,
-which can be assigned a string - and if assigned, is automatically displayed in
-formatted tracebacks.
+therefore propose to add:
+
+- a new method ``BaseException.add_note(note, *, replace=False)``,
+- ``BaseException.__notes__``, a read-only field which is a tuple of zero or
+  more note strings, and
+- support in the builtin traceback formatting code such that notes are
+  displayed in the formatted traceback following the exception string.
 
 
 Example usage
@@ -58,7 +63,7 @@ Example usage
    >>> try:
    ...     raise TypeError('bad type')
    ... except Exception as e:
-   ...     e.__note__ = 'Add some information'
+   ...     e.add_note('Add some information')
    ...     raise
    ...
    Traceback (most recent call last):
@@ -114,36 +119,41 @@ includes a note of the minimal failing example::
 
 Non-goals
 ---------
-``__note__`` is *not* intended to carry structured data.  If your note is for
-use by a program rather than display to a human, `we recommend
+Tracking multiple notes as a tuple, rather than by concatenating strings when
+notes are added, is intended to maintain the distinction between the
+individual notes. This might be required in specialized use cases, such
+as translation of the notes by packages like ``friendly-traceback``.
+
+However, ``__notes__`` is *not* intended to carry structured data.  If your
+note is for use by a program rather than display to a human, `we recommend
 <https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/26>`__
 instead (or additionally) choosing a convention for an attribute, e.g.
 ``err._parse_errors = ...`` on the error or ``ExceptionGroup``.
 
-As a rule of thumb, prefer `exception chaining
+As a rule of thumb, we suggest that you should prefer `exception chaining
 <https://docs.python.org/3/tutorial/errors.html#exception-chaining>`__ when the
 error is going to be re-raised or handled as an individual error, and prefer
-``__note__`` when you are collecting multiple exception objects to handle
-together or later. [1]_
+``.add_note()`` when you want to avoid changing the exception type or
+are collecting multiple exception objects to handle together. [1]_
 
 
 Specification
 =============
 
-``BaseException`` gains a new mutable attribute ``__note__``, which defaults to
-``None`` and may have a string assigned.  When an exception with a note is
-displayed, the note is displayed immediately after the exception.
-
-Assigning a new string value overrides an existing note; if concatenation is
-desired users are responsible for implementing it with e.g.::
-
-    e.__note__ = msg if e.__note__ is None else e.__note__ + "\n" + msg
+``BaseException`` gains a new read-only attribute ``__notes__``, an initially
+empty tuple, and a new method ``.add_note(note: str | None, *, replace:
+bool=False)``. If ``note`` is not ``None``, it is added to the exception's
+notes which appear in the standard traceback after the exception string. If
+``replace`` is true, all previously existing notes are removed before the new
+one is added. To clear all notes, use ``add_note(None, replace=True)``. A
+``TypeError`` is raised if ``note`` is neither a string nor ``None``.
 
-It is an error to assign a non-string-or-``None`` value to ``__note__``, or to
-attempt to delete the attribute.
+When an exception is displayed by the interpreter's builtin traceback-rendering code,
+its notes (if there are any) appear immediately after the exception message, in the order
+in which they were added, with each note starting on a new line.
 
 ``BaseExceptionGroup.subgroup`` and ``BaseExceptionGroup.split`` copy the
-``__note__`` of the original exception group to the parts.
+``__notes__`` of the original exception group to the parts.
 
 
 Backwards Compatibility
@@ -153,27 +163,33 @@ System-defined or "dunder" names (following the pattern ``__*__``) are part of
 the language specification, with `unassigned names reserved for future use and
 subject to breakage without warning
 <https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers>`__.
+We are also unaware of any code which *would* be broken by adding ``__notes__``.
 
-We are also unaware of any code which *would* be broken by adding ``__note__``;
-assigning to a ``.__note__`` attribute already *works* on current versions of
-Python - the note just won't be displayed with the traceback and exception
-message.
-
+We were also unable to find any code which would be broken by the addition of
+``BaseException.add_note()``: while searching Google and `GitHub finds several
+definitions <https://grep.app/search?q=.add_note%28&filter[lang][0]=Python>`__
+of an ``.add_note()`` method, none of them are on a subclass of
+``BaseException``.
 
 
 How to Teach This
 =================
 
-The ``__note__`` attribute will be documented as part of the language standard,
-and explained as part of `the "Errors and Exceptions" tutorial
-<https://github.com/python/cpython/pull/30441>`__.
+The ``add_note()`` method and ``__notes__`` attribute will be documented as
+part of the language standard, and explained as part of `the "Errors and
+Exceptions" tutorial <https://github.com/python/cpython/pull/30441>`__.
 
 
 Reference Implementation
 ========================
 
-``BaseException.__note__`` was `implemented in <https://github.com/python/cpython/pull/29880>`__ and released in CPython
-3.11.0a3, following discussions related to :pep:`654`. [2]_
+Following discussions related to :pep:`654` [2]_, an early version of this
+proposal was `implemented in <https://github.com/python/cpython/pull/29880>`__
+and released in CPython 3.11.0a3, with a mutable string-or-none ``__note__``
+attribute.
+
+`CPython PR #31317 <https://github.com/python/cpython/pull/31317>`__
+implements ``.add_note()`` and ``__notes__``.
 
 
 Rejected Ideas
@@ -189,8 +205,8 @@ or merely significant difficulties working out which explanation corresponds to
 which error. The new ``ExceptionGroup`` type intensifies these existing
 challenges.
 
-Keeping the ``__note__`` attached to the exception object, like the traceback,
-eliminates these problems.
+Keeping the ``__notes__`` attached to the exception object, in the same way as
+the ``__traceback__`` attribute, eliminates these problems.
 
 
 ``raise Wrapper(explanation) from err``
@@ -212,7 +228,7 @@ but not for libraries which call user code.
 Second, exception chaining reports several lines of additional detail, which
 are distracting for experienced users and can be very confusing for beginners.
 For example, six of the eleven lines reported for this simple example relate to
-exception chaining, and are unnecessary with ``BaseException.__note__``:
+exception chaining, and are unnecessary with ``BaseException.add_note()``:
 
 .. code-block:: python
 
@@ -242,19 +258,30 @@ exception chaining, and are unnecessary with ``BaseException.__note__``:
     You can reproduce this error by ...
 
 **In cases where these two problems do not apply, we encourage use of exception
-chaining rather than** ``__note__``.
+chaining rather than** ``__notes__``.
+
 
+A mutable ``__note__`` attribute
+--------------------------------
+The first draft and implementation of this PEP defined a single attribute
+``__note__``, which defaulted to ``None`` but could have a string assigned.
+This is substantially simpler if, and only if, there is at most one note.
 
-Subclass Exception and add ``__note__`` downstream
+To promote interoperability and support translation of error messages by
+libraries such as ``friendly-traceback``, without resorting to dubious parsing
+heuristics, we therefore settled on the ``.add_note()``-and-``__notes__`` API.
+
+
+Subclass Exception and add note support downstream
 --------------------------------------------------
 Traceback printing is built into the C code, and reimplemented in pure Python
-in traceback.py. To get ``err.__note__`` printed from a downstream
+in ``traceback.py``. To get ``err.__notes__`` printed from a downstream
 implementation would *also* require writing custom traceback-printing code;
 while this could be shared between projects and reuse some pieces of
 traceback.py we prefer to implement this once, upstream.
 
 Custom exception types could implement their ``__str__`` method to include our
-proposed ``__note__`` semantics, but this would be rarely and inconsistently
+proposed ``__notes__`` semantics, but this would be rarely and inconsistently
 applicable.
 
 
@@ -265,8 +292,8 @@ how to associate messages with the nested exceptions in ``ExceptionGroup`` s,
 such as a list of notes or mapping of exceptions to notes.  However, this would
 force a remarkably awkward API and retains a lesser form of the
 cross-referencing problem discussed under "use ``print()``" above; if this PEP
-is rejected we prefer the status quo. Finally, of course, ``__note__`` is not
-only useful with ``ExceptionGroup`` s!
+is rejected we prefer the status quo. Finally, of course, ``__notes__`` are
+not only useful with ``ExceptionGroup``\ s!
 
 
 
@@ -275,22 +302,15 @@ Possible Future Enhancements
 
 In addition to rejected alternatives, there have been a range of suggestions
 which we believe should be deferred to a future version, when we have more
-experience with the uses (and perhaps misuses) of ``__note__``.
+experience with the uses (and perhaps misuses) of ``__notes__``.
 
 
-Allow any object, and cast to string for display
-------------------------------------------------
+Allow any object, and convert to string for display
+---------------------------------------------------
 We have not identified any scenario where libraries would want to do anything
 but either concatenate or replace notes, and so the additional complexity and
 interoperability challenges do not seem justified.
 
-Permitting any object would also force any future structured API to change the
-behaviour of already-legal code, whereas expanding the permitted contents of
-``__note__`` from strings to include other objects is fully
-backwards-compatible.  In the absence of any proposed use-case (see also
-`Non-goals`_), we prefer to begin with a restrictive API that can be relaxed
-later.
-
 We also note that converting an object to a string may raise an exception.
 It's more helpful for the traceback to point to the location where the note is
 attached to the exception, rather than where the exception and note are being
@@ -307,14 +327,12 @@ as it can be added as an enhancement later.
 
 .. code-block:: python
 
-    @contextlib.contextmanager def add_exc_note(note: str):
+    @contextlib.contextmanager
+    def add_exc_note(note: str):
         try:
             yield
         except Exception as err:
-            if err.__note__ is None:
-                err.__note__ = note
-            else:
-                err.__note__ = err.__note__ + "\n\n" + note
+            err.add_note(note)
             raise
 
     with add_exc_note(f"While attempting to frobnicate {item=}"):
@@ -330,6 +348,17 @@ does not address the original motivation of compatibility with
 Furthermore, we do not believe that the problem we are solving requires or
 justifies new language syntax.
 
+
+Acknowledgements
+================
+We wish to thank the many people who have assisted us through conversation,
+code review, design advice, and implementation: Adam Turner, Alex Grönholm,
+André Roberge, Barry Warsaw, Brett Cannon, CAM Gerlach, Carol Willing, Damian,
+Erlend Aasland, Gregory Smith, Guido van Rossum, Irit Katriel, Jelle Zijlstra,
+Ken Jin, Kumar Aditya, Mark Shannon, Matti Picus, Petr Viktorin,
+and pseudonymous commenters on Discord and Reddit.
+
+
 References
 ==========