From 0ca1bef0b125f2b1936a1e7dc8217928b258d465 Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Mon, 25 Oct 2021 17:56:54 +0100 Subject: [PATCH 1/7] [PEP 646] Explicitly allow *args: *Tuple[*Ts, T] --- pep-0646.rst | 36 ++++++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index 7cf08d45619..b7a7a8a7aae 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -422,8 +422,9 @@ must be of the type annotated. That is, if we specify ``*args`` to be type ``int then *all* arguments must be of type ``int``. This limits our ability to specify the type signatures of functions that take heterogeneous argument types. -If ``*args`` is annotated as a type variable tuple, however, the types of the -individual arguments become the types in the type variable tuple: +If ``*args`` is annotated as an *unpacked* type variable tuple, however, the +types of the individual arguments become the types in the type variable tuple. +For example: :: @@ -436,6 +437,25 @@ individual arguments become the types in the type variable tuple: If no arguments are passed, the type variable tuple behaves like an empty tuple, ``Tuple[()]``. +It is also possible to annotate ``*args`` as an unpacked tuple parameterised +using a type variable tuple. This enables referral to suffixes of the variadic +argument list. For example, in type stubs it might be desirable to specify +different overload behaviour depending on the final argument: [#args]_ + +:: + + # Suppose Tensor t has shape (2, 3, 5). + # reshape(t, (2, 15)) would combine the last two dimensions to form a + # new Tensor with shape (2, 15). This can also be done with + # reshape(t, (2, -1)). + + @overload + def reshape(tensor, *shape: *Tuple[*Ts, Literal[-1]]):... + + @overload + def reshape(tensor, *shape: *Ts): ... + + Note that, in keeping with the rule that type variable tuples must always be used unpacked, annotating ``*args`` as being a plain type variable tuple instance is *not* allowed: @@ -446,8 +466,9 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. -If ``*args`` itself is annotated as ``Tuple[*Ts]``, the old behaviour still applies: -all arguments must be a ``Tuple`` parameterised with the same types. +If ``*args`` itself is annotated as an *unpacked* ``Tuple[*Ts]``, the old +behaviour still applies: all arguments must be a ``Tuple`` parameterised with +the same types. :: @@ -1293,6 +1314,13 @@ References .. [#syntax-proposal] Matthew Rahtz et al., 'Shape annotation syntax proposal': https://docs.google.com/document/d/1But-hjet8-djv519HEKvBN6Ik2lW3yu0ojZo6pG9osY/edit +.. [#args] For this particular example to be useful, we we would need + additional type operators such as ``Multiply`` and ``Divide``, as + proposed in Tensor Typing Open Design Meeting: + https://docs.google.com/presentation/d/16VGNfVo6Kxk9lBhm1cx1-Jr19dqESyy10WkhtMgUeDY/preview. + See also the discussion about whether type suffixing itself is + important at https://mail.python.org/archives/list/typing-sig@python.org/thread/3RFXH6C3HB5XL65ZZFXKIZP4EPE4B4PK/. + .. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/ From d42763588c7a8d93c564908aaf930c24cc8a4de8 Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Wed, 27 Oct 2021 11:56:07 +0100 Subject: [PATCH 2/7] PEP 646: Switch to execle for the *Tuple[*Ts, T] example --- pep-0646.rst | 35 +++++++++++------------------------ 1 file changed, 11 insertions(+), 24 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index b7a7a8a7aae..82dcdb4c024 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -422,7 +422,7 @@ must be of the type annotated. That is, if we specify ``*args`` to be type ``int then *all* arguments must be of type ``int``. This limits our ability to specify the type signatures of functions that take heterogeneous argument types. -If ``*args`` is annotated as an *unpacked* type variable tuple, however, the +If ``*args`` is annotated as a star-prefixed type variable tuple, however, the types of the individual arguments become the types in the type variable tuple. For example: @@ -437,24 +437,18 @@ For example: If no arguments are passed, the type variable tuple behaves like an empty tuple, ``Tuple[()]``. -It is also possible to annotate ``*args`` as an unpacked tuple parameterised +It is also possible to annotate ``*args`` as a star-prefix tuple parameterised using a type variable tuple. This enables referral to suffixes of the variadic -argument list. For example, in type stubs it might be desirable to specify -different overload behaviour depending on the final argument: [#args]_ +argument list. For example: :: - # Suppose Tensor t has shape (2, 3, 5). - # reshape(t, (2, 15)) would combine the last two dimensions to form a - # new Tensor with shape (2, 15). This can also be done with - # reshape(t, (2, -1)). - - @overload - def reshape(tensor, *shape: *Tuple[*Ts, Literal[-1]]):... - - @overload - def reshape(tensor, *shape: *Ts): ... + # os.execle takes arguments 'path, arg0, arg1, ..., env' + def execle(path: str, *args: *Tuple[*Ts, Mapping]) → None: ... + # Note that this is different to + def execle(path: str, *args: *Ts, env: Mapping) → None: ... + # which would make `env` a keyword-only argument. Note that, in keeping with the rule that type variable tuples must always be used unpacked, annotating ``*args`` as being a plain type variable tuple @@ -466,9 +460,9 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. -If ``*args`` itself is annotated as an *unpacked* ``Tuple[*Ts]``, the old -behaviour still applies: all arguments must be a ``Tuple`` parameterised with -the same types. + +If ``*args`` itself is annotated as ``Tuple[*Ts]``, the old behaviour still applies: +all arguments must be a ``Tuple`` parameterised with the same types. :: @@ -1314,13 +1308,6 @@ References .. [#syntax-proposal] Matthew Rahtz et al., 'Shape annotation syntax proposal': https://docs.google.com/document/d/1But-hjet8-djv519HEKvBN6Ik2lW3yu0ojZo6pG9osY/edit -.. [#args] For this particular example to be useful, we we would need - additional type operators such as ``Multiply`` and ``Divide``, as - proposed in Tensor Typing Open Design Meeting: - https://docs.google.com/presentation/d/16VGNfVo6Kxk9lBhm1cx1-Jr19dqESyy10WkhtMgUeDY/preview. - See also the discussion about whether type suffixing itself is - important at https://mail.python.org/archives/list/typing-sig@python.org/thread/3RFXH6C3HB5XL65ZZFXKIZP4EPE4B4PK/. - .. [#arbitrary_len] Discussion on Python typing-sig mailing list: https://mail.python.org/archives/list/typing-sig@python.org/thread/SQVTQYWIOI4TIO7NNBTFFWFMSMS2TA4J/ From 219e60319efe50e78689ec663c1d4c4496e656cd Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Wed, 27 Oct 2021 11:57:25 +0100 Subject: [PATCH 3/7] PEP 646: Fix typos --- pep-0646.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index 82dcdb4c024..beb56cffa52 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -423,8 +423,7 @@ then *all* arguments must be of type ``int``. This limits our ability to specify the type signatures of functions that take heterogeneous argument types. If ``*args`` is annotated as a star-prefixed type variable tuple, however, the -types of the individual arguments become the types in the type variable tuple. -For example: +types of the individual arguments become the types in the type variable tuple: :: @@ -437,7 +436,7 @@ For example: If no arguments are passed, the type variable tuple behaves like an empty tuple, ``Tuple[()]``. -It is also possible to annotate ``*args`` as a star-prefix tuple parameterised +It is also possible to annotate ``*args`` as a star-prefixed tuple parameterised using a type variable tuple. This enables referral to suffixes of the variadic argument list. For example: @@ -460,7 +459,6 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. - If ``*args`` itself is annotated as ``Tuple[*Ts]``, the old behaviour still applies: all arguments must be a ``Tuple`` parameterised with the same types. From 20a3750ce65574e481c95c8d2db722eb7f086656 Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Thu, 28 Oct 2021 10:17:20 +0100 Subject: [PATCH 4/7] PEP 646: Fix implicit Any; 'suffixes' -> 'prefixes and suffixes' --- pep-0646.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index beb56cffa52..9a051536bdb 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -437,16 +437,16 @@ If no arguments are passed, the type variable tuple behaves like an empty tuple, ``Tuple[()]``. It is also possible to annotate ``*args`` as a star-prefixed tuple parameterised -using a type variable tuple. This enables referral to suffixes of the variadic -argument list. For example: +using a type variable tuple. This enables referral to prefixes or suffixes of +the variadic argument list. For example: :: # os.execle takes arguments 'path, arg0, arg1, ..., env' - def execle(path: str, *args: *Tuple[*Ts, Mapping]) → None: ... + def execle(path: str, *args: *Tuple[*Ts, Mapping[str, str]]) → None: ... # Note that this is different to - def execle(path: str, *args: *Ts, env: Mapping) → None: ... + def execle(path: str, *args: *Ts, env: Mapping[str, str]) → None: ... # which would make `env` a keyword-only argument. Note that, in keeping with the rule that type variable tuples must always From 8cb676b7bb2442b57559e8fb11f3d41fd05e6831 Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Mon, 8 Nov 2021 19:03:44 +0000 Subject: [PATCH 5/7] PEP 646: Explain *args: *Tuple[int, int] and similar; reorganise *args section --- pep-0646.rst | 86 ++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 60 insertions(+), 26 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index 9a051536bdb..e8b2c3acb3f 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -414,16 +414,25 @@ Normal ``TypeVar`` instances can also be prefixed and/or suffixed: z = prefix_tuple(x=0, y=(True, 'a')) # Inferred type of z is Tuple[int, bool, str] -``*args`` as a Type Variable Tuple ----------------------------------- +Heterogeneous ``*args`` Types +----------------------------- PEP 484 states that when a type annotation is provided for ``*args``, every argument must be of the type annotated. That is, if we specify ``*args`` to be type ``int``, then *all* arguments must be of type ``int``. This limits our ability to specify the type signatures of functions that take heterogeneous argument types. -If ``*args`` is annotated as a star-prefixed type variable tuple, however, the -types of the individual arguments become the types in the type variable tuple: +With the new use for the star operator introduced in this PEP, however, we +can do better. In short, if ``*args`` is annotated as a star-prefixed type +tuple or type variable tuple, the types of the individual arguments are bound +to the types within the star-prefixed container. + +Using a Type Variable Tuple +''''''''''''''''''''''''''' + +For example, annotating ``*args`` as a star-prefixed type variable tuple +results in the types in the type variable tuple being bound to the types +of the individual arguments: :: @@ -436,19 +445,6 @@ types of the individual arguments become the types in the type variable tuple: If no arguments are passed, the type variable tuple behaves like an empty tuple, ``Tuple[()]``. -It is also possible to annotate ``*args`` as a star-prefixed tuple parameterised -using a type variable tuple. This enables referral to prefixes or suffixes of -the variadic argument list. For example: - -:: - - # os.execle takes arguments 'path, arg0, arg1, ..., env' - def execle(path: str, *args: *Tuple[*Ts, Mapping[str, str]]) → None: ... - - # Note that this is different to - def execle(path: str, *args: *Ts, env: Mapping[str, str]) → None: ... - # which would make `env` a keyword-only argument. - Note that, in keeping with the rule that type variable tuples must always be used unpacked, annotating ``*args`` as being a plain type variable tuple instance is *not* allowed: @@ -459,8 +455,37 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. -If ``*args`` itself is annotated as ``Tuple[*Ts]``, the old behaviour still applies: -all arguments must be a ``Tuple`` parameterised with the same types. + +Also note that, following `Type Variable Tuples Must Have Known Length`_, +the following should *not* type-check as valid (even though it is, of course, +valid at runtime): + +:: + + def foo(*args: *Ts): ... + + def bar(x: Tuple[int, ...]): + foo(*x) # NOT valid + +Using a Tuple of Types +'''''''''''''''''''''' + +Similarly, ``*args`` can be annotated as a star-prefixed type tuple: + +:: + + def foo(*args: *Tuple[int, int]): ... + # Very similar to: + def bar(arg1: int, arg2: int): ... + # But a) `foo` may be more convenient if e.g. passing `args` on + # to something else, and b) `foo` disallows keyword arguments. + +Note that using an arbitrary-length type tuple, e.g. ``*args: *Tuple[int, ...]``, +is *not* valid here as it would be exactly the same as ``*args: int``. + +Also note that if ``*args`` is annotated as an *unstarred* ``Tuple[*Ts]``, the +old ``*args`` behaviour still applies: all arguments must be a ``Tuple`` parameterised +with the same types. :: @@ -470,18 +495,27 @@ all arguments must be a ``Tuple`` parameterised with the same types. foo((0,), (1, 2)) # Error foo((0,), ('1',)) # Error -Following `Type Variable Tuples Must Have Known Length`_, note -that the following should *not* type-check as valid (even though it is, of -course, valid at runtime): +Using a Type Variable Tuple Inside a Tuple +'''''''''''''''''''''''''''''''''''''''''' + +Finally, by using a type variable tuple inside a tuple of other +types, we can refer to prefixes or suffixes of the variadic +argument list. For example: :: - def foo(*args: *Ts): ... + # os.execle takes arguments 'path, arg0, arg1, ..., env' + def execle(path: str, *args: *Tuple[*Ts, Mapping[str, str]]) → None: ... - def bar(x: Tuple[int, ...]): - foo(*x) # NOT valid + # Note that this is different to + def execle(path: str, *args: *Ts, env: Mapping[str, str]) → None: ... + # which would make `env` a keyword-only argument. + +What About ``**kwargs?`` +'''''''''''''''''''''''' -Finally, note that a type variable tuple may *not* be used as the type of +The PEP does *not* introduce any new behaviour for typing ``**kwargs``. +In particular, a type variable tuple may *not* be used as the type of ``**kwargs``. (We do not yet know of a use case for this feature, so we prefer to leave the ground fresh for a potential future PEP.) From f88946c894fe0be41bf48ed822bab1fce4b404e7 Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Tue, 9 Nov 2021 17:35:22 +0000 Subject: [PATCH 6/7] PEP 646: Don't explicitly disallow passing an arbitrary-sized tuple to a function with signature '*args: *Ts' --- pep-0646.rst | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index e8b2c3acb3f..cdd4c6353c6 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -456,17 +456,6 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. -Also note that, following `Type Variable Tuples Must Have Known Length`_, -the following should *not* type-check as valid (even though it is, of course, -valid at runtime): - -:: - - def foo(*args: *Ts): ... - - def bar(x: Tuple[int, ...]): - foo(*x) # NOT valid - Using a Tuple of Types '''''''''''''''''''''' From 7be542314bae9b72beb67a97cb2841ae58f140ee Mon Sep 17 00:00:00 2001 From: Matthew Rahtz Date: Thu, 11 Nov 2021 10:57:02 +0000 Subject: [PATCH 7/7] PEP 646: Allow *args: *Tuple[int, ...] --- pep-0646.rst | 45 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/pep-0646.rst b/pep-0646.rst index cdd4c6353c6..e48b985f1e0 100644 --- a/pep-0646.rst +++ b/pep-0646.rst @@ -456,25 +456,46 @@ instance is *not* allowed: ``*args`` is the only case where an argument can be annotated as ``*Ts`` directly; other arguments should use ``*Ts`` to parameterise something else, e.g. ``Tuple[*Ts]``. +Also note that, following `Type Variable Tuples Must Have Known Length`_, +the following should *not* type-check as valid (even though it is, of +course, valid at runtime): + +:: + + def foo(*args: *Ts): ... + + def bar(x: Tuple[int, ...]): + foo(*x) # NOT valid + Using a Tuple of Types '''''''''''''''''''''' -Similarly, ``*args`` can be annotated as a star-prefixed type tuple: +Similarly, ``*args`` can be annotated as a star-prefixed tuple type: :: def foo(*args: *Tuple[int, int]): ... - # Very similar to: + +This is very similar to: + +:: + def bar(arg1: int, arg2: int): ... - # But a) `foo` may be more convenient if e.g. passing `args` on - # to something else, and b) `foo` disallows keyword arguments. -Note that using an arbitrary-length type tuple, e.g. ``*args: *Tuple[int, ...]``, -is *not* valid here as it would be exactly the same as ``*args: int``. +However, ``bar`` is subtly different than ``foo``: a) ``foo`` may be more +convenient if e.g. passing ``args`` on to something else, and b) ``foo`` +disallows keyword arguments. + +Note that arbitrary-length tuple types can also be used here: + +:: + + def foo(*args: *Tuple[int, ...]): ... + +This is roughly the same as ``*args: int``. Also note that if ``*args`` is annotated as an *unstarred* ``Tuple[*Ts]``, the -old ``*args`` behaviour still applies: all arguments must be a ``Tuple`` parameterised -with the same types. +old ``*args`` behaviour still applies: all arguments must be a ``Tuple`` parameterised with the same types. :: @@ -496,9 +517,13 @@ argument list. For example: # os.execle takes arguments 'path, arg0, arg1, ..., env' def execle(path: str, *args: *Tuple[*Ts, Mapping[str, str]]) → None: ... - # Note that this is different to +Note this this is different to + +:: + def execle(path: str, *args: *Ts, env: Mapping[str, str]) → None: ... - # which would make `env` a keyword-only argument. + +as this would make ``env`` a keyword-only argument. What About ``**kwargs?`` ''''''''''''''''''''''''