Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic UID allocation #3600

Merged
merged 49 commits into from
Nov 29, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
836573a
Dynamically allocate UIDs
edolstra Oct 31, 2017
c3e0a68
canonicalisePathMetaData(): Support a UID range
edolstra May 14, 2020
f5fa3de
Run builds in their own cgroup
edolstra May 14, 2020
ca2f64b
Reduce # of UIDs per build to 65536
edolstra May 14, 2020
7bdcf43
Destroy the cgroup prior to building
edolstra May 16, 2020
570c443
Simplify cgroup creation
edolstra May 16, 2020
ba50c3e
Add "uid-range" and "systemd-cgroup" system features
edolstra May 19, 2020
8c4cce5
Fix macOS build
edolstra May 20, 2020
7349f25
Only mount /sys in uid-range builds
edolstra May 20, 2020
2546c63
Merge commit 'f66bbd8c7bb1472facf8917e58e3cd4f6ddfa1b5' into auto-uid…
Ericson2314 Oct 17, 2020
81c5f75
Merge commit '6aa64627c8e431c3b187f7bb44c943d06e39b929' into auto-uid…
Ericson2314 Oct 17, 2020
2c9a8e7
Merge commit '97ffc1e0139e124b7e36b5d1a62b90300f231118' into auto-uid…
Ericson2314 Oct 17, 2020
801e6d9
Merge commit 'd334fd48824b41b57e267cd2926fa9619b7718e3' into auto-uid…
Ericson2314 Oct 17, 2020
d3cfc14
Merge remote-tracking branch 'upstream/master' into auto-uid-allocation
Ericson2314 Oct 17, 2020
e023c98
Merge remote-tracking branch 'upstream/master' into auto-uid-allocation
Ericson2314 Apr 27, 2021
3b82c1a
Merge remote-tracking branch 'upstream/master' into auto-uid-allocation
mkenigs Sep 15, 2021
79152e3
Merge pull request #5212 from mkenigs/auto-uid-allocation
edolstra Sep 15, 2021
dc92b01
Merge remote-tracking branch 'upstream/master' into auto-uid-allocation
Ericson2314 Feb 28, 2022
47dec82
Merge pull request #6181 from obsidiansystems/auto-uid-allocation
thufschmitt Mar 1, 2022
b95facc
Merge remote-tracking branch 'origin/master' into auto-uid-allocation
edolstra Nov 3, 2022
6259fd7
Fix indentation
edolstra Nov 4, 2022
40911d7
Remove stray tab
edolstra Nov 4, 2022
2fde7e0
Split auto UID allocation from cgroups
edolstra Nov 8, 2022
05d2586
Fix build on macOS
edolstra Nov 8, 2022
6c6eff8
Remove the SystemdCgroup feature
edolstra Nov 10, 2022
f423d44
Fix segfault in unprivileged mode
edolstra Nov 17, 2022
f1ab082
createTempDir(): Use std::atomic
edolstra Nov 18, 2022
128910b
Separate cgroup support from auto-uid-allocation
edolstra Nov 18, 2022
20f66c6
Indentation
edolstra Nov 18, 2022
fa68eb3
Get CPU stats from the cgroup
edolstra Nov 18, 2022
e6b71f8
Use cgroup.kill to quickly kill cgroups
edolstra Nov 18, 2022
f538ee4
Rename derivedPathsWithHintsToJSON -> builtPathsToJSON
edolstra Nov 21, 2022
300753d
nix build --json: Include build statistics
edolstra Nov 21, 2022
ec45f4b
Fix indentation
edolstra Nov 21, 2022
653b32a
Merge remote-tracking branch 'origin/master' into auto-uid-allocation
edolstra Nov 21, 2022
82d5cf2
Fix macOS build
edolstra Nov 21, 2022
9d17ce0
AutoUserLock: If sandboxing is disabled, use the build users group
edolstra Nov 21, 2022
c776dfb
Use hex for startId
edolstra Nov 21, 2022
b37c2d8
Always call setgroups()
edolstra Nov 22, 2022
3d23b9d
SimpleUserLock::getSupplementaryGIDs(): Filter out main gid
edolstra Nov 22, 2022
6292d56
Merge remote-tracking branch 'origin/master' into auto-uid-allocation
edolstra Nov 23, 2022
989fc8a
Add release notes
edolstra Nov 23, 2022
2aa3f2e
Include UID in hex
edolstra Nov 23, 2022
f1b5c68
Add tests for auto-uid-allocation, uid-range and cgroups
edolstra Nov 27, 2022
fc14585
Fix evaluation
edolstra Nov 27, 2022
ff12d1c
Check that auto-allocated UIDs don't clash with existing accounts
edolstra Nov 28, 2022
67bcb99
Add a setting for enabling cgroups
edolstra Nov 28, 2022
7dd3e1f
Add example
edolstra Nov 28, 2022
4f762e2
Restore ownership of / for non-uid-range builds
edolstra Nov 29, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions doc/manual/src/release-notes/rl-next.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,45 @@
This avoids a lot of spurious errors where some benign strings end-up having
a context just because they are read from a store path
([#7260](https://github.com/NixOS/nix/pull/7260)).

* Nix can now automatically pick UIDs for builds, removing the need to
create `nixbld*` user accounts. These UIDs are allocated starting at
872415232 (0x34000000) on Linux and 56930 on macOS.

This is an experimental feature. To enable it, add the following to
`nix.conf`:

```
extra-experimental-features = auto-allocate-uids
auto-allocate-uids = true
```

* On Linux, Nix can now run builds in a user namespace where the build
runs as root (UID 0) and has 65,536 UIDs available. This is
primarily useful for running containers such as `systemd-nspawn`
inside a Nix build. For an example, see
https://github.com/NixOS/nix/blob/67bcb99700a0da1395fa063d7c6586740b304598/tests/systemd-nspawn.nix.

A build can enable this by requiring the `uid-range` system feature,
i.e. by setting the derivation attribute

```
requiredSystemFeatures = [ "uid-range" ];
```

The `uid-range` system feature requires the `auto-allocate-uids`
setting to be enabled (see above).

* On Linux, Nix has experimental support for running builds inside a
cgroup. It can be enabled by adding

```
extra-experimental-features = cgroups
use-cgroups = true
```

to `nix.conf`. Cgroups are required for derivations that require the
`uid-range` system feature.

* `nix build --json` now prints some statistics about top-level
derivations, such as CPU statistics when cgroups are enabled.
6 changes: 6 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,12 @@
overlay = self.overlays.default;
});

tests.containers = (import ./tests/containers.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
overlay = self.overlays.default;
});

tests.setuid = nixpkgs.lib.genAttrs
["i686-linux" "x86_64-linux"]
(system:
Expand Down
2 changes: 1 addition & 1 deletion src/libcmd/installables.cc
Original file line number Diff line number Diff line change
Expand Up @@ -931,7 +931,7 @@ std::vector<std::pair<std::shared_ptr<Installable>, BuiltPathWithResult>> Instal

case Realise::Outputs: {
if (settings.printMissing)
printMissing(store, pathsToBuild, lvlInfo);
printMissing(store, pathsToBuild, lvlInfo);

for (auto & buildResult : store->buildPathsWithResults(pathsToBuild, bMode, evalStore)) {
if (!buildResult.success())
Expand Down
5 changes: 4 additions & 1 deletion src/libstore/build-result.hh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

#include <string>
#include <chrono>

#include <optional>

namespace nix {

Expand Down Expand Up @@ -78,6 +78,9 @@ struct BuildResult
was repeated). */
time_t startTime = 0, stopTime = 0;

/* User and system CPU time the build took. */
std::optional<std::chrono::microseconds> cpuUser, cpuSystem;

bool success()
{
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
Expand Down
8 changes: 8 additions & 0 deletions src/libstore/build/derivation-goal.cc
Original file line number Diff line number Diff line change
Expand Up @@ -887,6 +887,14 @@ void DerivationGoal::buildDone()

cleanupPostChildKill();

if (buildResult.cpuUser && buildResult.cpuSystem) {
debug("builder for '%s' terminated with status %d, user CPU %.3fs, system CPU %.3fs",
worker.store.printStorePath(drvPath),
status,
((double) buildResult.cpuUser->count()) / 1000000,
((double) buildResult.cpuSystem->count()) / 1000000);
}

bool diskFull = false;

try {
Expand Down
Loading