Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-95023: Added os.setns and os.unshare to easily switch between namespaces on Linux #95046

Merged
merged 48 commits into from
Oct 20, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
0bdd8bc
gh-95023: added os.setns and os.unshare for namespaces switching on L…
noamcohen97 Jul 20, 2022
3685a27
remove gil release
noamcohen97 Jul 20, 2022
8f22740
better setns, unshare doc
noamcohen97 Jul 21, 2022
5115c8a
added example to setns
noamcohen97 Jul 21, 2022
8845a86
added note about fileno for setns
noamcohen97 Jul 21, 2022
d7bb582
added see also section for os.unshare
noamcohen97 Jul 21, 2022
987613a
Update configure.ac
noamcohen97 Jul 26, 2022
4c91413
Revert "remove gil release"
noamcohen97 Jul 26, 2022
7d23963
better docs
noamcohen97 Jul 26, 2022
b7abf20
assume Linux platform has a working readlink
noamcohen97 Jul 26, 2022
af74db5
change ifdefs
noamcohen97 Jul 26, 2022
5181126
added missing `CLONE_*` consts
noamcohen97 Jul 26, 2022
3a37ac2
run unshare in a different process
noamcohen97 Jul 26, 2022
1386833
reformat doc
noamcohen97 Jul 26, 2022
ae4b661
indent test code
noamcohen97 Jul 26, 2022
57b2c84
Apply suggestions from code review
noamcohen97 Jul 26, 2022
b2df7f7
better doc
noamcohen97 Jul 26, 2022
b4a68b6
Merge branch 'main' into namespaces
noamcohen97 Jul 26, 2022
dc51d01
remove whitespaces from doc
noamcohen97 Jul 26, 2022
5a3cff5
fix NEWS entry
noamcohen97 Jul 26, 2022
1d45196
move test code to another file
noamcohen97 Jul 26, 2022
84c4b8c
add glibc requirements to doc
noamcohen97 Jul 26, 2022
b9d3a34
Revert "move test code to another file"
noamcohen97 Jul 26, 2022
51c60d4
proofreaders review fixes
noamcohen97 Jul 26, 2022
225e06b
fix doc whitespaces
noamcohen97 Jul 26, 2022
fb64bb7
better documentation
noamcohen97 Jul 27, 2022
01d4af4
fix docs
noamcohen97 Jul 27, 2022
15e6d8b
remove bitwise or explanation
noamcohen97 Jul 27, 2022
375165b
Apply suggestions from code review
noamcohen97 Jul 28, 2022
7e2b44c
fix typo
noamcohen97 Jul 28, 2022
3ae952c
ref
noamcohen97 Jul 28, 2022
432d274
fix doc whitespace
noamcohen97 Jul 28, 2022
afa9a00
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Jul 28, 2022
54c7bce
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Jul 28, 2022
01713ec
regen posixmodule
noamcohen97 Jul 28, 2022
a6bb345
add name to `Misc/ACKS`
noamcohen97 Jul 28, 2022
1c0fb83
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Aug 17, 2022
dac402a
fix doc availability
noamcohen97 Aug 17, 2022
ff7f961
regen global strings
noamcohen97 Aug 17, 2022
cc44f01
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Oct 7, 2022
58bc8a6
Merge branch 'main' into namespaces
noamcohen97 Oct 7, 2022
b14396e
Merge remote-tracking branch 'upstream/main' into namespaces
noamcohen97 Oct 7, 2022
50c4809
Merge branch 'main' into namespaces
noamcohen97 Oct 14, 2022
de7dd3d
do not use stdout for test
noamcohen97 Oct 17, 2022
5c1bbdd
move all test login to subprocess
noamcohen97 Oct 19, 2022
29129d9
Update Lib/test/test_posix.py
noamcohen97 Oct 20, 2022
94883a4
add comments to tests
noamcohen97 Oct 20, 2022
3e365d2
bugfix in test while handling `ENOSPC`
noamcohen97 Oct 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions Doc/library/os.rst
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,28 @@ process and user.
See the documentation for :func:`getgroups` for cases where it may not
return the same group list set by calling setgroups().

.. function:: setns(fd, nstype=0)

Reassociate thread with a namespace.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
If *fd* refers to a ``/proc/[pid]/ns/`` link, :func:`setns` reassociates the
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
calling thread with the namespace associated with that link, subject to any
constraints imposed by the *nstype* argument (or any if 0).
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
Since Linux 5.8, *fd* may refer to a PID file descriptor obtained from
:c:func:`pidfd_open`. In this case :func:`setns` reassociates the calling thread
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
into one or more of the same namespaces as the thread referred to by *fd* subject
to any constraints imposed by the *nstype* which is a bit mask specified by ORing together one or more of the ``CLONE_NEW*`` constants. the caller's memberships in unspecified namespaces are left unchanged.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
*fd* can be any object with a :meth:`fileno()` method, or a raw file descriptor.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

This example reassociates the thread with the ``init`` process' network namespace::
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

fd = os.open("/proc/1/ns/net", os.O_RDONLY)
os.setns(fd, os.CLONE_NEWNET)
os.close(fd)

.. availability:: Linux 3.0 or newer.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

.. versionadded:: 3.12

noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
.. function:: setpgrp()

Call the system call :c:func:`setpgrp` or ``setpgrp(0, 0)`` depending on
Expand Down Expand Up @@ -732,6 +754,37 @@ process and user.
The function is now always available and is also available on Windows.


.. function:: unshare(flags)

Disassociate parts of the process execution context.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
The *flags* argument is a bit mask that specifies which parts of the execution
context should be unshared. This argument is specified by ORing together zero
or more of the ``CLONE_*`` constants.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
If *flags* is specified as zero, no changes are made to the calling process's
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
execution context.

.. availability:: Linux 2.6.16 or newer.

.. versionadded:: 3.12

.. seealso::

The :func:`.setns` function.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
Flags to the :func:`unshare` function, if the implementation supports them.
See the Linux manual for the exact effect and availability.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

.. data:: CLONE_FS
CLONE_FILES
CLONE_NEWNS
CLONE_NEWCGROUP
CLONE_NEWUTS
CLONE_NEWIPC
CLONE_NEWUSER
CLONE_NEWPID
CLONE_NEWNET
CLONE_NEWTIME

.. _os-newstreams:

File Object Creation
Expand Down
29 changes: 29 additions & 0 deletions Lib/test/test_posix.py
Original file line number Diff line number Diff line change
Expand Up @@ -2172,6 +2172,35 @@ def test_utime(self):
os.utime("path", dir_fd=0)


class NamespacesTests(unittest.TestCase):
"""Tests for os.unshare() and os.setns()."""

@unittest.skipUnless(hasattr(os, 'unshare'), 'needs os.unshare()')
@unittest.skipUnless(hasattr(os, 'setns'), 'needs os.setns()')
@unittest.skipUnless(hasattr(os, 'readlink'), 'needs os.readlink()')
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
@unittest.skipUnless(os.path.exists('/proc/self/ns/uts'), 'need /proc/self/ns/uts')
@support.requires_linux_version(3, 0, 0)
def test_unshare_setns(self):
original = os.readlink('/proc/self/ns/uts')
original_fd = os.open('/proc/self/ns/uts', os.O_RDONLY)
self.addCleanup(os.close, original_fd)

try:
os.unshare(os.CLONE_NEWUTS)
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
except OSError as e:
self.assertEqual(e.errno, errno.EPERM)
self.skipTest("unprivileged users cannot call unshare.")
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

current = os.readlink('/proc/self/ns/uts')
self.assertNotEqual(original, current)

self.assertRaises(OSError, os.setns, original_fd, os.CLONE_NEWNET)
os.setns(original_fd, os.CLONE_NEWUTS)

current = os.readlink('/proc/self/ns/uts')
self.assertEqual(original, current)


def tearDownModule():
support.reap_children()

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Implement :func:`os.setns` and :func:`os.unshare` for Linux. Patch by Noam Cohen
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
97 changes: 96 additions & 1 deletion Modules/clinic/posixmodule.c.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

80 changes: 80 additions & 0 deletions Modules/posixmodule.c
Original file line number Diff line number Diff line change
Expand Up @@ -8588,6 +8588,49 @@ os_pidfd_open_impl(PyObject *module, pid_t pid, unsigned int flags)
#endif


#if defined(__linux__) && defined(HAVE_SETNS)
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
/*[clinic input]
os.setns
fd: fildes
nstype: int = 0

Allows the calling thread to move into different namespaces.
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
[clinic start generated code]*/

static PyObject *
os_setns_impl(PyObject *module, int fd, int nstype)
/*[clinic end generated code: output=5dbd055bfb66ecd0 input=c097c9aa123c43ce]*/
{
if (setns(fd, nstype) != 0) {
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
return posix_error();
}

Py_RETURN_NONE;
}
#endif


#if defined(__linux__) && defined(HAVE_UNSHARE)
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
/*[clinic input]
os.unshare
flags: int

Allows a process (or thread) to disassociate parts of its execution context.
[clinic start generated code]*/

static PyObject *
os_unshare_impl(PyObject *module, int flags)
/*[clinic end generated code: output=1b3177906dd237ee input=f8d7bd2c69325537]*/
{
if (unshare(flags) != 0) {
return posix_error();
}

Py_RETURN_NONE;
}
#endif


#if defined(HAVE_READLINK) || defined(MS_WINDOWS)
/*[clinic input]
os.readlink
Expand Down Expand Up @@ -14930,6 +14973,8 @@ static PyMethodDef posix_methods[] = {
OS__ADD_DLL_DIRECTORY_METHODDEF
OS__REMOVE_DLL_DIRECTORY_METHODDEF
OS_WAITSTATUS_TO_EXITCODE_METHODDEF
OS_SETNS_METHODDEF
OS_UNSHARE_METHODDEF
{NULL, NULL} /* Sentinel */
};

Expand Down Expand Up @@ -15375,6 +15420,41 @@ all_ins(PyObject *m)
#ifdef SCHED_FX
if (PyModule_AddIntConstant(m, "SCHED_FX", SCHED_FSS)) return -1;
#endif

/* constants for namespaces */
#if defined(__linux__) && (defined(HAVE_SETNS) || defined(HAVE_UNSHARE))
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
#ifdef CLONE_FS
if (PyModule_AddIntMacro(m, CLONE_FS)) return -1;
#endif
#ifdef CLONE_FILES
if (PyModule_AddIntMacro(m, CLONE_FILES)) return -1;
#endif
#ifdef CLONE_NEWNS
if (PyModule_AddIntMacro(m, CLONE_NEWNS)) return -1;
#endif
#ifdef CLONE_NEWCGROUP
if (PyModule_AddIntMacro(m, CLONE_NEWCGROUP)) return -1;
#endif
#ifdef CLONE_NEWUTS
if (PyModule_AddIntMacro(m, CLONE_NEWUTS)) return -1;
#endif
#ifdef CLONE_NEWIPC
if (PyModule_AddIntMacro(m, CLONE_NEWIPC)) return -1;
#endif
#ifdef CLONE_NEWUSER
if (PyModule_AddIntMacro(m, CLONE_NEWUSER)) return -1;
#endif
#ifdef CLONE_NEWPID
if (PyModule_AddIntMacro(m, CLONE_NEWPID)) return -1;
#endif
#ifdef CLONE_NEWNET
if (PyModule_AddIntMacro(m, CLONE_NEWNET)) return -1;
#endif
#ifdef CLONE_NEWTIME
if (PyModule_AddIntMacro(m, CLONE_NEWTIME)) return -1;
#endif
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved
#endif

#endif

#ifdef USE_XATTRS
Expand Down
14 changes: 14 additions & 0 deletions configure

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -4937,6 +4937,9 @@ AC_CHECK_FUNCS(setpgrp,
[])
)

# check for namespace functions
AC_CHECK_FUNCS(setns unshare)
noamcohen97 marked this conversation as resolved.
Show resolved Hide resolved

dnl We search for both crypt and crypt_r as one or the other may be defined
dnl libxcrypt provides <crypt.h> and libcrypt with crypt_r() since
dnl at least 3.1.1 from 2015.
Expand Down
6 changes: 6 additions & 0 deletions pyconfig.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -1019,6 +1019,9 @@
/* Define to 1 if you have the `setlocale' function. */
#undef HAVE_SETLOCALE

/* Define to 1 if you have the `setns' function. */
#undef HAVE_SETNS

/* Define to 1 if you have the `setpgid' function. */
#undef HAVE_SETPGID

Expand Down Expand Up @@ -1383,6 +1386,9 @@
/* Define to 1 if you have the `unlinkat' function. */
#undef HAVE_UNLINKAT

/* Define to 1 if you have the `unshare' function. */
#undef HAVE_UNSHARE

/* Define if you have a useable wchar_t type defined in wchar.h; useable means
wchar_t must be an unsigned type with at least 16 bits. (see
Include/unicodeobject.h). */
Expand Down