Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-95023: Added os.setns and os.unshare to easily switch between namespaces on Linux #95046

Merged
merged 48 commits into from
Oct 20, 2022
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0bdd8bc
gh-95023: added os.setns and os.unshare for namespaces switching on L…
noamcohen97 Jul 20, 2022
3685a27
remove gil release
noamcohen97 Jul 20, 2022
8f22740
better setns, unshare doc
noamcohen97 Jul 21, 2022
5115c8a
added example to setns
noamcohen97 Jul 21, 2022
8845a86
added note about fileno for setns
noamcohen97 Jul 21, 2022
d7bb582
added see also section for os.unshare
noamcohen97 Jul 21, 2022
987613a
Update configure.ac
noamcohen97 Jul 26, 2022
4c91413
Revert "remove gil release"
noamcohen97 Jul 26, 2022
7d23963
better docs
noamcohen97 Jul 26, 2022
b7abf20
assume Linux platform has a working readlink
noamcohen97 Jul 26, 2022
af74db5
change ifdefs
noamcohen97 Jul 26, 2022
5181126
added missing `CLONE_*` consts
noamcohen97 Jul 26, 2022
3a37ac2
run unshare in a different process
noamcohen97 Jul 26, 2022
1386833
reformat doc
noamcohen97 Jul 26, 2022
ae4b661
indent test code
noamcohen97 Jul 26, 2022
57b2c84
Apply suggestions from code review
noamcohen97 Jul 26, 2022
b2df7f7
better doc
noamcohen97 Jul 26, 2022
b4a68b6
Merge branch 'main' into namespaces
noamcohen97 Jul 26, 2022
dc51d01
remove whitespaces from doc
noamcohen97 Jul 26, 2022
5a3cff5
fix NEWS entry
noamcohen97 Jul 26, 2022
1d45196
move test code to another file
noamcohen97 Jul 26, 2022
84c4b8c
add glibc requirements to doc
noamcohen97 Jul 26, 2022
b9d3a34
Revert "move test code to another file"
noamcohen97 Jul 26, 2022
51c60d4
proofreaders review fixes
noamcohen97 Jul 26, 2022
225e06b
fix doc whitespaces
noamcohen97 Jul 26, 2022
fb64bb7
better documentation
noamcohen97 Jul 27, 2022
01d4af4
fix docs
noamcohen97 Jul 27, 2022
15e6d8b
remove bitwise or explanation
noamcohen97 Jul 27, 2022
375165b
Apply suggestions from code review
noamcohen97 Jul 28, 2022
7e2b44c
fix typo
noamcohen97 Jul 28, 2022
3ae952c
ref
noamcohen97 Jul 28, 2022
432d274
fix doc whitespace
noamcohen97 Jul 28, 2022
afa9a00
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Jul 28, 2022
54c7bce
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Jul 28, 2022
01713ec
regen posixmodule
noamcohen97 Jul 28, 2022
a6bb345
add name to `Misc/ACKS`
noamcohen97 Jul 28, 2022
1c0fb83
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Aug 17, 2022
dac402a
fix doc availability
noamcohen97 Aug 17, 2022
ff7f961
regen global strings
noamcohen97 Aug 17, 2022
cc44f01
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Oct 7, 2022
58bc8a6
Merge branch 'main' into namespaces
noamcohen97 Oct 7, 2022
b14396e
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Oct 7, 2022
50c4809
Merge branch 'main' into namespaces
noamcohen97 Oct 14, 2022
de7dd3d
do not use stdout for test
noamcohen97 Oct 17, 2022
5c1bbdd
move all test login to subprocess
noamcohen97 Oct 19, 2022
29129d9
Update Lib/test/test_posix.py
noamcohen97 Oct 20, 2022
94883a4
add comments to tests
noamcohen97 Oct 20, 2022
3e365d2
bugfix in test while handling `ENOSPC`
noamcohen97 Oct 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions Doc/library/os.rst
Original file line number Diff line number Diff line change
Expand Up @@ -590,6 +590,44 @@ process and user.
See the documentation for :func:`getgroups` for cases where it may not
return the same group list set by calling setgroups().

.. function:: setns(fd, nstype=0)

Reassociate the current thread with a Linux namespace.
See the :manpage:`setns(2)` and :manpage:`namespaces(7)` man pages for more
details.

If *fd* refers to a :file:`/proc/{pid}/ns/` link, ``setns()`` reassociates the
calling thread with the namespace associated with that link,
and *nstype* may be set to one of the
:ref:`CLONE_NEW* constants <os-unshare-clone-flags>`
to impose constraints on the operation
(``0`` means no constraints).

Since Linux 5.8, *fd* may refer to a PID file descriptor obtained from
:func:`~os.pidfd_open`. In this case, ``setns()`` reassociates the calling thread
into one or more of the same namespaces as the thread referred to by *fd*.
This is subject to any constraints imposed by *nstype*,
which is a bit mask combining one or more of the
:ref:`CLONE_NEW* constants <os-unshare-clone-flags>`,
e.g. ``setns(fd, os.CLONE_NEWUTS | os.CLONE_NEWPID)``.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
The caller's memberships in unspecified namespaces are left unchanged.

*fd* can be any object with a :meth:`~io.IOBase.fileno` method, or a raw file descriptor.

This example reassociates the thread with the ``init`` process's network namespace::

fd = os.open("/proc/1/ns/net", os.O_RDONLY)
os.setns(fd, os.CLONE_NEWNET)
os.close(fd)

.. availability:: Linux >= 3.0 with glibc >= 2.14.

.. versionadded:: 3.12

noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
.. seealso::

The :func:`~os.unshare` function.

.. function:: setpgrp()

Call the system call :c:func:`setpgrp` or ``setpgrp(0, 0)`` depending on
Expand Down Expand Up @@ -756,6 +794,49 @@ process and user.
The function is now always available and is also available on Windows.


.. function:: unshare(flags)

Disassociate parts of the process execution context, and move them into a
newly created namespace.
See the :manpage:`unshare(2)`
man page for more details.
The *flags* argument is a bit mask, combining zero or more of the
:ref:`CLONE_* constants <os-unshare-clone-flags>`,
that specifies which parts of the execution context should be
unshared from their existing associations and moved to a new namespace.
If the *flags* argument is ``0``, no changes are made to the calling process's
execution context.

.. availability:: Linux >= 2.6.16.

.. versionadded:: 3.12

.. seealso::

The :func:`~os.setns` function.

noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
.. _os-unshare-clone-flags:

Flags to the :func:`unshare` function, if the implementation supports them.
See :manpage:`unshare(2)` in the Linux manual
for their exact effect and availability.

.. data:: CLONE_FILES
CLONE_FS
CLONE_NEWCGROUP
CLONE_NEWIPC
CLONE_NEWNET
CLONE_NEWNS
CLONE_NEWPID
CLONE_NEWTIME
CLONE_NEWUSER
CLONE_NEWUTS
CLONE_SIGHAND
CLONE_SYSVSEM
CLONE_THREAD
CLONE_VM


.. _os-newstreams:

File Object Creation
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -489,6 +489,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(node_depth)
STRUCT_FOR_ID(node_offset)
STRUCT_FOR_ID(ns)
STRUCT_FOR_ID(nstype)
STRUCT_FOR_ID(number)
STRUCT_FOR_ID(obj)
STRUCT_FOR_ID(object)
Expand Down
7 changes: 7 additions & 0 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions Lib/test/test_posix.py
Original file line number Diff line number Diff line change
Expand Up @@ -2194,6 +2194,46 @@ def test_utime(self):
os.utime("path", dir_fd=0)


class NamespacesTests(unittest.TestCase):
"""Tests for os.unshare() and os.setns()."""

@unittest.skipUnless(hasattr(os, 'unshare'), 'needs os.unshare()')
@unittest.skipUnless(hasattr(os, 'setns'), 'needs os.setns()')
@unittest.skipUnless(os.path.exists('/proc/self/ns/uts'), 'need /proc/self/ns/uts')
@support.requires_linux_version(3, 0, 0)
def test_unshare_setns(self):
code = """if 1:
import errno
import os
fd = os.open('/proc/self/ns/uts', os.O_RDONLY)
try:
original = os.readlink('/proc/self/ns/uts')
try:
os.unshare(os.CLONE_NEWUTS)
except OSError as e:
if e.errno not in (errno.ENOSPC,):
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
raise
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
new = os.readlink('/proc/self/ns/uts')
if original == new:
raise Exception('os.unshare failed')
os.setns(fd, os.CLONE_NEWUTS)
restored = os.readlink('/proc/self/ns/uts')
if original != restored:
raise Exception('os.setns failed')
except PermissionError:
# The calling process did not have the required privileges
# for this operation
pass
except OSError as e:
if e.errno not in (errno.ENOSYS, errno.EINVAL, errno.ENOMEM):
raise
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
finally:
os.close(fd)
"""

assert_python_ok("-c", code)


def tearDownModule():
support.reap_children()

Expand Down
1 change: 1 addition & 0 deletions Misc/ACKS
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ Hervé Coatanhay
Riccardo Coccioli
Nick Coghlan
Josh Cogliati
Noam Cohen
Dave Cole
Terrence Cole
Benjamin Collar
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implement :func:`os.setns` and :func:`os.unshare` for Linux. Patch by Noam Cohen.
151 changes: 150 additions & 1 deletion Modules/clinic/posixmodule.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading