Better specify number of nested scopes

[RoscoeRHiggins]

BPO	26205
Nosy	@terryjreedy, @ezio-melotti, @eryksun, @vedgar, @Mariatta, @csabella, @miss-islington, @nanjekyejoannah
PRs	bpo-26205: Specify the number of nested scopes #21324 [3.9] bpo-26205: Specify the number of nested scopes (GH-21324) #21341 [3.8] bpo-26205: Specify the number of nested scopes (GH-21324) #21342

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/Mariatta'
closed_at = None
created_at = <Date 2016-01-26.02:16:27.767>
labels = ['type-feature', '3.8', '3.9', '3.10', 'docs']
title = 'Better specify number of nested scopes'
updated_at = <Date 2020-11-04.21:36:36.791>
user = 'https://bugs.python.org/RoscoeRHiggins'

bugs.python.org fields:

activity = <Date 2020-11-04.21:36:36.791>
actor = 'brett.cannon'
assignee = 'Mariatta'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2016-01-26.02:16:27.767>
creator = 'Roscoe R. Higgins'
dependencies = []
files = []
hgrepos = []
issue_num = 26205
keywords = ['patch']
message_count = 20.0
messages = ['258941', '258964', '258978', '258980', '259228', '340000', '373022', '373048', '373049', '373055', '373056', '373058', '373059', '373061', '373062', '373063', '373068', '373085', '373087', '373125']
nosy_count = 11.0
nosy_names = ['terry.reedy', 'ezio.melotti', 'docs@python', 'eryksun', 'veky', 'Hibou57', 'Roscoe R. Higgins', 'Mariatta', 'cheryl.sabella', 'miss-islington', 'nanjekyejoannah']
pr_nums = ['21324', '21341', '21342']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26205'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

• PR: gh-70393: Clarify mention of "middle" scope #98839

• PR: [3.11] gh-70393: Clarify mention of "middle" scope (GH-98839) #100186

• PR: [3.10] gh-70393: Clarify mention of "middle" scope (GH-98839) #100187

[RoscoeRHiggins]

In chapter 9. Classes of the Python3.5 documentation it says:

"At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:",

followed by a list containing 4 items.
Further down a middle scope is mentioned (although mentioned by name). This was confusing for a while.

[brettcannon]

https://docs.python.org/3/tutorial/classes.html for the docs that Roscoe is talking about.

So the sentence is technically correct, it just takes careful reading to grasp what's being said. There are "at least three nested scopes", but there can be *up to* four scopes. Since "the scopes of any enclosing functions" is not necessarily true for all code scopes, you end up with at least three, but possibly four scopes.

Obviously the wording could be clearer, so if you want to sign the CLA, Roscoe, and propose a rewording that would be appreciated!

[Hibou57]

Better at least two, if I'm not wrong: the innermost scope may be the module's scope. So there is always at least the module scope and the built‑ins scope, at least two up to four.

(if I have not overlooked something)

[brettcannon]

It depends on how you want to view things as to whether you can claim there are two or three scopes for module-level code. While it's true that the local scope operates just like global scope, to Python it's more like local == global rather than the local scope simply doesn't exist (hence why locals() never throws an exception saying the scope doesn't exist but instead is the same as globals()).

The "three scope" phrasing also predates nested scopes from back when Python had its LGB scoping rules: Local-Global-Builtin. Back then we just said Python had three scopes and left it at that since it was basically always true and didn't confuse anyone.

At this point I'm fine with just removing the number from the sentence and saying something like "At any time during execution, there are various nested scopes whose namespaces are directly accessible:".

[terryjreedy]

Would 'three or more' be any clearer than 'at least three'? They mean the same, but the first seems better to me in this context.

The real problem with this section are a) the use of Guido's first person 'I' and b) statements that were not changed when nested scope were added, but should have been.

"If a name is declared global, then all references and assignments go directly to the middle scope containing the module’s global names."

The global scope is no longer the middle scope. Roscoe pointed at this. With that removed, the sentence says that if a name is declared global, assignments go to global scope. This would be more meaningful if prefixed by a revised version of the following, which is several paragraphs down.

"A special quirk of Python is that – if no global statement is in effect – assignments to names always go into the innermost scope."

The special quirk part should go; 'global' would now have to be 'global or nonlocal', but I now think the following, preceeding the revised 'global' sentence above, would be better.

"By default, assignments to names always go into the innermost, local, namespace."

In other words, I think the sentence Roscoe flagged is the least of the problems with this section.

[csabella]

Assigning to @Mariatta for the sprints.

[vedgar]

Just to point out: it can be more than four, right? If you have a function aa within a function bb within a function cc, then bb's and cc's scope are different, even though functionally equivalent from the perspective of aa.

[terryjreedy]

At any one time, the language and interpreter sees the local scopes of enclosing functions as collectively 'nonlocal' for the purpose of accessing and rebinding. If there are bindings of a name 'x' in multiple enclosing local scopes, the binding for 'x' in the synthesized 'nonlocal' is the innermost one.

[vedgar]

How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."? Any ChainMap presents itself as a dictionary to the outside. That doesn't mean it doesn't have an internal structure.

Yes, nonlocal picks the innermost one because of its semantics, but it still needs to search through the scopes in order to know which one to pick, right?

[terryjreedy]

Vedran: "How is that different from saying that "At any one time, the language and interpreter sees all the scopes as one scope for the purpose of accessing and rebinding."?"

For access, one may bypass the default staged access by specifying a specific namespace. For rebinding elsewhere than in locals, one must specify one of the others. So the 4 scopes are each visible. However, the internal 'structure' of nonlocals is intentionally not visible or accessible. (The proposal to make them somehow accessible was considered and rejected.)
---

The PR replaces 'at least three' with 'various'. This is needlessly vague. There are always 3 namespaces and sometimes a 4th, so I think 'at least three' should be replaced with the specific '3 or 4'.

I rechecked two special cases. Exception clauses with 'as name' bind name in the current local namespace and auto delete the binding when done. Comprehensions bind the name and iterator of the 'for' clause in a temporary new local namespace, making the existing namespace temporarily part of a nonlocal namespace, which may or may not have been present before. '3 or 4' still applies in both cases.

[nanjekyejoannah]

PR updated with your change. @terry

[nanjekyejoannah]

New changeset 9ed3cd8 by Joannah Nanjekye in branch 'master':
bpo-26205: Specify the number of nested scopes (GH-21324)
9ed3cd8

[nanjekyejoannah]

Merged, this can be closed if there is consesus.

[nanjekyejoannah]

New changeset 7ceb3e3 by Miss Islington (bot) in branch '3.8':
bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21342)
7ceb3e3

[terryjreedy]

I wait for backports to be merged before closing. Someone has to approve them or manually merge them before that happens, so leaving issues open until all merges are done prevents totally forgetting about them.

For doc issues, the patch, if not drastically wrong, becomes the new status quo. So I think any further proposed change should normally be a new issue. So once the backports *are* done, you can close.

For code issues, 'drastically wrong' usually means that the patch failed on one of the buildbots, or fairly soon somewhere else, and needs either reversion or a hot fix.

[nanjekyejoannah]

New changeset 3f4a9fd by Miss Islington (bot) in branch '3.9':
bpo-26205: Specify the number of nested scopes (GH-21324) (GH-21341)
3f4a9fd

[vedgar]

Well, then "what name is in which namespace" is relative to which function we're considering. In my example above, imagine aa has x declared, bb has y, and cc has z. Then y and z are in the same namespace when we look from the perspective of aa, but they are not in the same namespace from the perspective of bb. Even worse, cc sees z but doesn't see y. How can they be in the same namespace then?

I always thought about namespaces as mappings from names to objects, independent of perspective. Whether two names are in the same namespace, should be a question with an objective answer.

[terryjreedy]

Since you ask, here is a extended summary of namespaces. There is one built-in namespace. There is one global namespace for each module, which is also the local namespace for top level code in that module. There is one local namespace for each class and function. There is one nonlocal namespace for each nested function. "At any time during execution" (the beginning of the doc sentence targeted by this issue), there are 3 or maybe 4 namespaces accessible with simple undotted names.

Module global and class local namespaces, including nested classes, are accessible from without the object with dotted names. There are rules for class and method code about access to superclass namespaces. Function locals are not necessarily accessible with standard python code, but at least in cpython, local names can be accessed via code objects. Default parameter values and current nonlocal values, when set, can be accessed via function objects. Dotted names are discussed elsewhere in the tutorial, while function and code object introspection is out of scope there. (And off topic for this issue ;-).

[terryjreedy]

Besides the sentence now revised, the initial post referenced confusion with 'middle scope' in "If a name is declared global, then all references and assignments go directly to the middle scope containing the module's global names."

This has not been directly discussed that I could find, but it seems inappropriate both at toplevel, where globals = locals, and in nested functions, where nonlocals are also 'middle'. Maybe replace "the middle scope containing the module's global names" with "the module's global namespace". The rest of this paragraph and the next could be reviewed.

[eryksun]

There is one local namespace for each class and function. There is one
nonlocal namespace for each nested function.

In the first sentence, replace "class" with "class statement" and "function" with "function call". The second sentence could use "nested-function call", but maybe rewrite it as, "There is one nonlocal namespace when a nested function is called."

[hauntsaninja]

#98839 for clarifying "middle" scope