Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pidfd: Cannot restore multiple processes pointing to a common dead pidfd #2496

Closed
bsach64 opened this issue Oct 18, 2024 · 2 comments · Fixed by #2505
Closed

pidfd: Cannot restore multiple processes pointing to a common dead pidfd #2496

bsach64 opened this issue Oct 18, 2024 · 2 comments · Fixed by #2505

Comments

@bsach64
Copy link
Member

bsach64 commented Oct 18, 2024 •
edited
Loading

Description
Please read the reproducer code: https://gist.github.com/bsach64/fcbdaec357fd7ed4212a86ac81b2e8bb

#define _GNU_SOURCE

#include <sys/statfs.h>
#include <stdio.h>
#include <sys/syscall.h>
#include <unistd.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <sys/stat.h>

#ifndef PID_FS_MAGIC
#define PID_FS_MAGIC 0x50494446
#endif

/*
 * main
 *	`- child
 *		`- grandchild
 *
 * main and child open a pidfd for grandchild.
 * Before C/R we kill grandchild.
 * We end up with two pidfds in different process of
 * pointing to a common dead process.
 */

static int pidfd_open(pid_t pid, unsigned int flags)
{
	return syscall(__NR_pidfd_open, pid, flags);
}

static int pidfd_send_signal(int pidfd, int sig, siginfo_t* info, unsigned int flags)
{
	return syscall(__NR_pidfd_send_signal, pidfd, sig, info, flags);
}

int main(int argc, char* argv[])
{
	#define READ 0
	#define WRITE 1

	int child, ret, gchild, p[2], status, pidfd;

	if (pipe(p)) {
		perror("pipe");
		return 1;
	}

	child = fork();
	if (child < 0) {
		perror("fork");
		return 1;
	} else if (child == 0) {
		int gchild = fork();
		close(p[READ]);
		if (gchild < 0) {
			perror("fork");
			return 1;
		} else if (gchild == 0) {
			close(p[WRITE]);
			while(1)
				sleep(1000);
		} else {
			int pidfd;
			pidfd = pidfd_open(gchild, 0);
			if (pidfd < 0) {
				perror("pidfd_open");
				return 1;
			}

			if (write(p[WRITE], &gchild, sizeof(int)) != sizeof(int)) {
				printf("write");
				return 1;
			}

			close(p[WRITE]);
			if (waitpid(gchild, &status, 0) != gchild) {
				printf("waitpid");
				return 1;
			}

			while(1)
				sleep(1000);
		}
	}

	close(p[WRITE]);
	if (read(p[READ], &gchild, sizeof(int)) != sizeof(int)) {
		printf("write");
		return 1;
	}
	close(p[READ]);

	pidfd = pidfd_open(gchild, 0);
	if (pidfd < 0) {
		perror("pidfd_open");
		return 1;
	}

	/*
	* We kill grandchild processes only after opening pidfds.
	*/
	if (pidfd_send_signal(pidfd, SIGKILL, NULL, 0)) {
		perror("pidfd_send_signal");
		return 1;
	}
       /* Wait for C/R */
	while(1)
		sleep(1000);
}

Relevant Restore Failure Logs

(00.026088)  65165: Error (criu/pidfd.c:177): pidfd: Could not wait on temporary process with pid: 65616: No child processes
(00.026094)  65165: Error (criu/pidfd.c:245): pidfd: Failed to delete dead_pidfd struct
(00.026101)  65165: Error (criu/pidfd.c:261): pidfd: Can't create pidfd 0x000006 NSpid: -1 flags: 0
(00.026126)  65165: Error (criu/files.c:1221): Unable to open fd=3 id=0x6
(00.026167)  65166: Restore via sigreturn
(00.026192) Error (criu/cr-restore.c:2314): Restoring FAILED.
(00.026789) Error (criu/cr-restore.c:1261): 65165 killed by signal 9: Killed

I think the issue stems from the fact that the process that creates the tmp process is not the which is waiting for it.
A different process may kill this tmp process, but the one that creates it should be the one that waits for it to exit.
I will try to raise a PR with the fix over the next couple of weeks.

CRIU logs and information:

CRIU full restore logs: 

(00.000000) CRIU run id = 0xeffffffc0001004d
(00.000090) Version: 3.18 (gitid 59afbf343)
(00.000103) Running on fedora Linux 6.10.12-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Sep 30 21:38:25 UTC 2024 x86_64
(00.000121) Loaded kdat cache from /run/criu.kdat
(00.000147) Hugetlb size 1024 Mb is supported but cannot get dev's number
(00.001057) cpu: x86_family 6 x86_vendor_id GenuineIntel x86_model_id 13th Gen Intel(R) Core(TM) i7-13700H
(00.001082) cpu: fpu: xfeatures_mask 0x205 xsave_size 2696 xsave_size_max 2696 xsaves_size 1664
(00.001090) cpu: fpu: x87 floating point registers     xstate_offsets      0 / 0      xstate_sizes    160 / 160   
(00.001095) cpu: fpu: AVX registers                    xstate_offsets    576 / 576    xstate_sizes    256 / 256   
(00.001099) cpu: fpu: Protection Keys User registers   xstate_offsets   2688 / 832    xstate_sizes      8 / 8     
(00.001131) rlimit: RLIMIT_NOFILE unlimited for self
(00.001278) cpu: fpu:1 fxsr:1 xsave:1 xsaveopt:1 xsavec:1 xgetbv1:1 xsaves:1
(00.001340) kernel pid_max=4194304
(00.001348) Reading image tree
(00.001385) Add mnt ns 6 pid 65165
(00.001391) Add net ns 2 pid 65165
(00.001395) Add pid ns 1 pid 65165
(00.001413) pstree pid_max=65166
(00.001425) Migrating process tree (SID 63706->63021)
(00.001430) Will restore in 0 namespaces
(00.001433) NS mask to use 0
(00.001472) Collecting 51/56 (flags 3)
(00.001481) No memfd.img image
(00.001485)  `- ... done
(00.001494) Collecting 40/54 (flags 2)
(00.001515) Collected [home/bsach/Code/criu-experiments/final/dead] ID 0x1
(00.001523) Collected [usr/lib64/libc.so.6] ID 0x2
(00.001528) Collected [usr/lib64/ld-linux-x86-64.so.2] ID 0x3
(00.001533) Collected [dev/pts/4] ID 0x5
(00.001547) pidfd: Collected : id 0x000006 flags 0 NSpid -1 ino 65467
(00.001559) Collected [home/bsach/Code/criu-experiments/final] ID 0x7
(00.001567) Collected [.] ID 0x8
(00.001572) pidfd: Collected : id 0x000009 flags 0 NSpid -1 ino 65467
(00.001578) Collected [home/bsach/Code/criu-experiments/final] ID 0xa
(00.001582) Collected [.] ID 0xb
(00.001588)  `- ... done
(00.001592) Collecting 46/69 (flags 0)
(00.001597) No remap-fpath.img image
(00.001601)  `- ... done
(00.001627) No apparmor.img image
(00.001673) cg: Preparing cgroups yard (cgroups restore mode 0x4)
(00.001864) cg: Opening .criu.cgyard.O6e57r as cg yard
(00.001884) cg: 	Making controller dir .criu.cgyard.O6e57r/unified (), type cgroup2
(00.001933) cg: Determined cgroup dir unified/user.slice/user-1000.slice/user@1000.service/app.slice/app-foot.slice/app-foot-63705.scope already exist
(00.001944) cg: Skip restoring properties on cgroup dir unified/user.slice/user-1000.slice/user@1000.service/app.slice/app-foot.slice/app-foot-63705.scope
(00.001972) cg: 	Making controller dir .criu.cgyard.O6e57r/zdtmtst (none,name=zdtmtst), type cgroup
(00.002130) cg: Determined cgroup dir zdtmtst/ already exist
(00.002137) cg: Skip restoring properties on cgroup dir zdtmtst/
(00.002159) cg: Determined cgroup dir zdtmtst//holder.8995a148-b632-4488-bc4e-13c6f56febd2 already exist
(00.002164) cg: Skip restoring properties on cgroup dir zdtmtst//holder.8995a148-b632-4488-bc4e-13c6f56febd2
(00.002180) cg: 	Making controller dir .criu.cgyard.O6e57r/zdtmtst.defaultroot (none,name=zdtmtst.defaultroot), type cgroup
(00.002262) cg: Determined cgroup dir zdtmtst.defaultroot/ already exist
(00.002269) cg: Skip restoring properties on cgroup dir zdtmtst.defaultroot/
(00.002287) cg: Determined cgroup dir zdtmtst.defaultroot//holder.8995a148-b632-4488-bc4e-13c6f56febd2 already exist
(00.002291) cg: Skip restoring properties on cgroup dir zdtmtst.defaultroot//holder.8995a148-b632-4488-bc4e-13c6f56febd2
(00.002639) Running pre-restore scripts
(00.002750) No pidns-1.img image
(00.002767) cg: cgroud: Daemon started
(00.002862) Forking task with 65165 pid (flags 0x0)
(00.002867) Creating process using clone3()
(00.003088) PID: real 65165 virt 65165
(00.003421)  65165: timens: monotonic -4 648245920
(00.003442)  65165: timens: boottime -4 648224914
(00.003533)  65165: cg: Move into 2
(00.003541)  65165: cg:   `-> unified//user.slice/user-1000.slice/user@1000.service/app.slice/app-foot.slice/app-foot-63705.scope/cgroup.procs
(00.020445)  65165: cg:   `-> zdtmtst///cgroup.procs
(00.020553)  65165: cg:   `-> zdtmtst.defaultroot///cgroup.procs
(00.020598)  65165: Calling restore_sid() for init
(00.020652)  65165: Collecting 44/37 (flags 2)
(00.020744)  65165: tty: Collected tty ID 0x4 (pts)
(00.020778)  65165:  `- ... done
(00.020787)  65165: Collecting 45/51 (flags 0)
(00.020800)  65165: No tty-data.img image
(00.020822)  65165:  `- ... done
(00.020829)  65165: Restoring namespaces 65165 flags 0x0
(00.020874)  65165: Preparing info about shared resources
(00.020912)  65165: Collecting 48/38 (flags 0)
(00.020926)  65165: No filelocks.img image
(00.020933)  65165:  `- ... done
(00.020939)  65165: Collecting 42/27 (flags 0)
(00.020949)  65165: No pipes-data.img image
(00.020956)  65165:  `- ... done
(00.020962)  65165: Collecting 43/27 (flags 0)
(00.020972)  65165: No fifo-data.img image
(00.020978)  65165:  `- ... done
(00.020984)  65165: Collecting 41/70 (flags 0)
(00.020994)  65165: No sk-queues.img image
(00.021001)  65165:  `- ... done
(00.021091)  65165: Found 21 VMAs in image
(00.021107)  65165: vma 0x400000 0x401000
(00.021116)  65165: vma 0x401000 0x402000
(00.021122)  65165: vma 0x402000 0x403000
(00.021128)  65165: vma 0x403000 0x404000
(00.021134)  65165: vma 0x404000 0x405000
(00.021140)  65165: vma 0x7f13d6e2c000 0x7f13d6e2f000
(00.021147)  65165: vma 0x7f13d6e2f000 0x7f13d6e57000
(00.021153)  65165: vma 0x7f13d6e57000 0x7f13d6fc4000
(00.021159)  65165: vma 0x7f13d6fc4000 0x7f13d7012000
(00.021165)  65165: vma 0x7f13d7012000 0x7f13d7016000
(00.021171)  65165: vma 0x7f13d7016000 0x7f13d7018000
(00.021178)  65165: vma 0x7f13d7018000 0x7f13d7022000
(00.021184)  65165: vma 0x7f13d7034000 0x7f13d7038000
(00.021190)  65165: vma 0x7f13d7038000 0x7f13d703a000
(00.021196)  65165: vma 0x7f13d703a000 0x7f13d703b000
(00.021202)  65165: vma 0x7f13d703b000 0x7f13d7063000
(00.021208)  65165: vma 0x7f13d7063000 0x7f13d706d000
(00.021214)  65165: vma 0x7f13d706d000 0x7f13d706f000
(00.021219)  65165: vma 0x7f13d706f000 0x7f13d7071000
(00.021225)  65165: vma 0x7ffcc683f000 0x7ffcc6860000
(00.021231)  65165: vma 0xffffffffff600000 0xffffffffff601000
(00.021258)  65165: Collect fdinfo pid=65165 fd=0 id=0x4
(00.021270)  65165: Collect fdinfo pid=65165 fd=1 id=0x4
(00.021279)  65165: Collect fdinfo pid=65165 fd=2 id=0x4
(00.021286)  65165: Collect fdinfo pid=65165 fd=3 id=0x6
(00.021394)  65165: Found 21 VMAs in image
(00.021407)  65165: vma 0x400000 0x401000
(00.021415)  65165: vma 0x401000 0x402000
(00.021421)  65165: vma 0x402000 0x403000
(00.021427)  65165: vma 0x403000 0x404000
(00.021433)  65165: vma 0x404000 0x405000
(00.021439)  65165: vma 0x7f13d6e2c000 0x7f13d6e2f000
(00.021446)  65165: vma 0x7f13d6e2f000 0x7f13d6e57000
(00.021452)  65165: vma 0x7f13d6e57000 0x7f13d6fc4000
(00.021458)  65165: vma 0x7f13d6fc4000 0x7f13d7012000
(00.021464)  65165: vma 0x7f13d7012000 0x7f13d7016000
(00.021470)  65165: vma 0x7f13d7016000 0x7f13d7018000
(00.021476)  65165: vma 0x7f13d7018000 0x7f13d7022000
(00.021482)  65165: vma 0x7f13d7034000 0x7f13d7038000
(00.021488)  65165: vma 0x7f13d7038000 0x7f13d703a000
(00.021494)  65165: vma 0x7f13d703a000 0x7f13d703b000
(00.021500)  65165: vma 0x7f13d703b000 0x7f13d7063000
(00.021506)  65165: vma 0x7f13d7063000 0x7f13d706d000
(00.021512)  65165: vma 0x7f13d706d000 0x7f13d706f000
(00.021518)  65165: vma 0x7f13d706f000 0x7f13d7071000
(00.021524)  65165: vma 0x7ffcc683f000 0x7ffcc6860000
(00.021530)  65165: vma 0xffffffffff600000 0xffffffffff601000
(00.021552)  65165: Collect fdinfo pid=65166 fd=0 id=0x4
(00.021562)  65165: Collect fdinfo pid=65166 fd=1 id=0x4
(00.021570)  65165: Collect fdinfo pid=65166 fd=2 id=0x4
(00.021577)  65165: Collect fdinfo pid=65166 fd=3 id=0x9
(00.021608)  65165: Found two COW VMAs @0x400000-0x401000
(00.021618)  65165: Found two COW VMAs @0x401000-0x402000
(00.021624)  65165: Found two COW VMAs @0x402000-0x403000
(00.021630)  65165: Found two COW VMAs @0x403000-0x404000
(00.021636)  65165: Found two COW VMAs @0x404000-0x405000
(00.021678)  65165: Found two COW VMAs @0x7f13d6e2c000-0x7f13d6e2f000
(00.021688)  65165: Found two COW VMAs @0x7f13d6e2f000-0x7f13d6e57000
(00.021694)  65165: Found two COW VMAs @0x7f13d6e57000-0x7f13d6fc4000
(00.021700)  65165: Found two COW VMAs @0x7f13d6fc4000-0x7f13d7012000
(00.021706)  65165: Found two COW VMAs @0x7f13d7012000-0x7f13d7016000
(00.021712)  65165: Found two COW VMAs @0x7f13d7016000-0x7f13d7018000
(00.021718)  65165: Found two COW VMAs @0x7f13d7018000-0x7f13d7022000
(00.021724)  65165: Found two COW VMAs @0x7f13d7034000-0x7f13d7038000
(00.021730)  65165: Found two COW VMAs @0x7f13d7038000-0x7f13d703a000
(00.021736)  65165: Found two COW VMAs @0x7f13d703a000-0x7f13d703b000
(00.021742)  65165: Found two COW VMAs @0x7f13d703b000-0x7f13d7063000
(00.021748)  65165: Found two COW VMAs @0x7f13d7063000-0x7f13d706d000
(00.021753)  65165: Found two COW VMAs @0x7f13d706d000-0x7f13d706f000
(00.021759)  65165: Found two COW VMAs @0x7f13d706f000-0x7f13d7071000
(00.021765)  65165: Found two COW VMAs @0x7ffcc683f000-0x7ffcc6860000
(00.021875)  65165: skqueue: Preparing SCMs
(00.021886)  65165: tty: Unpaired slave 4
(00.021894)  65165: tty: ctl tty leader 0x4
(00.021900)  65165: tty: Inherit terminal for id 0x4
(00.021907)  65165: tty: head driver pts id 0x4 index 4 (master 0 sid 63706 pgrp 65165 inherit 1)
(00.021917)  65165: tty: Found orphan slave fake leader (0x4)
(00.021931)  65165: unix: ghost: Resolving addresses
(00.021939)  65165: File descs:
(00.021945)  65165:  `- type 1 ID 0x1
(00.021951)  65165:  `- type 1 ID 0x2
(00.021957)  65165:  `- type 1 ID 0x3
(00.021963)  65165:  `- type 11 ID 0x4
(00.021969)  65165:    `- FD 0 pid 65165
(00.021975)  65165:    `- FD 1 pid 65165
(00.021981)  65165:    `- FD 2 pid 65165
(00.021986)  65165:    `- FD 0 pid 65166
(00.021992)  65165:    `- FD 1 pid 65166
(00.021998)  65165:    `- FD 2 pid 65166
(00.022004)  65165:  `- type 1 ID 0x5
(00.022009)  65165:  `- type 20 ID 0x6
(00.022015)  65165:    `- FD 3 pid 65165
(00.022021)  65165:  `- type 1 ID 0x7
(00.022027)  65165:  `- type 1 ID 0x8
(00.022033)  65165:  `- type 20 ID 0x9
(00.022039)  65165:    `- FD 3 pid 65166
(00.022044)  65165:  `- type 1 ID 0xa
(00.022050)  65165:  `- type 1 ID 0xb
(00.022340)  65165: No parent images directory provided
(00.022515)  65165: Opened local page read 1 (parent 0)
(00.022664)  65165: 	premap 0x00000000400000-0x00000000401000 -> 00007f16cf600000
(00.022682)  65165: 	premap 0x00000000401000-0x00000000402000 -> 00007f16cf601000
(00.022693)  65165: 	premap 0x00000000402000-0x00000000403000 -> 00007f16cf602000
(00.022704)  65165: 	premap 0x00000000403000-0x00000000404000 -> 00007f16cf603000
(00.022715)  65165: 	premap 0x00000000404000-0x00000000405000 -> 00007f16cf604000
(00.022725)  65165: 	premap 0x007f13d6e2c000-0x007f13d6e2f000 -> 00007f16cf605000
(00.022791)  65165: 	premap 0x007f13d6e2f000-0x007f13d6e57000 -> 00007f16cf608000
(00.022804)  65165: 	premap 0x007f13d6e57000-0x007f13d6fc4000 -> 00007f16cf630000
(00.022818)  65165: 	premap 0x007f13d6fc4000-0x007f13d7012000 -> 00007f16cf79d000
(00.022845)  65165: 	premap 0x007f13d7012000-0x007f13d7016000 -> 00007f16cf7eb000
(00.022856)  65165: 	premap 0x007f13d7016000-0x007f13d7018000 -> 00007f16cf7ef000
(00.022867)  65165: 	premap 0x007f13d7018000-0x007f13d7022000 -> 00007f16cf7f1000
(00.022877)  65165: 	premap 0x007f13d7034000-0x007f13d7038000 -> 00007f16cf7fb000
(00.022887)  65165: 	premap 0x007f13d7038000-0x007f13d703a000 -> 00007f16cf7ff000
(00.022937)  65165: 	premap 0x007f13d703a000-0x007f13d703b000 -> 00007f16cf801000
(00.022951)  65165: 	premap 0x007f13d703b000-0x007f13d7063000 -> 00007f16cf802000
(00.022966)  65165: 	premap 0x007f13d7063000-0x007f13d706d000 -> 00007f16cf82a000
(00.022996)  65165: 	premap 0x007f13d706d000-0x007f13d706f000 -> 00007f16cf834000
(00.023016)  65165: 	premap 0x007f13d706f000-0x007f13d7071000 -> 00007f16cf836000
(00.023028)  65165: 	premap 0x007ffcc683f000-0x007ffcc6860000 -> 00007f16cf838000
(00.023035)  65165: pr65165-1 Read 401000 1 pages
(00.023043)  65165: pr65165-1 Read 403000 1 pages
(00.023089)  65165: pr65165-1 Read 404000 1 pages
(00.023095)  65165: pr65165-1 Read 7f13d6e2c000 2 pages
(00.023100)  65165: pr65165-1 Read 7f13d7012000 4 pages
(00.023105)  65165: pr65165-1 Read 7f13d7016000 2 pages
(00.023110)  65165: pr65165-1 Read 7f13d701c000 5 pages
(00.023115)  65165: pr65165-1 Read 7f13d7038000 2 pages
(00.023120)  65165: pr65165-1 Read 7f13d706d000 2 pages
(00.023125)  65165: pr65165-1 Read 7f13d706f000 2 pages
(00.023130)  65165: pr65165-1 Read 7ffcc685c000 4 pages
(00.023135)  65165: Read piov iovs 7, from 0, len 106496, first 0x7f16cf601000:4096
(00.023334)  65165: nr_restored_pages: 26
(00.023341)  65165: nr_shared_pages:   0
(00.023346)  65165: nr_dropped_pages:  0
(00.023350)  65165: nr_enqueued:       0
(00.023354)  65165: nr_lazy:           0
(00.023361)  65165: Restore on-core sigactions for 65165
(00.023419)  65165: Restoring children in alien sessions:
(00.023425)  65165: Restoring children in our session:
(00.023578)  65165: Forking task with 65166 pid (flags 0x0)
(00.023584)  65165: Creating process using clone3()
(00.024204)  65165: Restoring 65165 to 65165 pgid
(00.024230)  65165: 	will call setpgid, mine pgid is 65613
(00.024369)  65165: PID: real 65166 virt 65166
(00.024476)  65166: cg: Cgroups 2 inherited from parent
(00.024581)  65166: No parent images directory provided
(00.024650)  65166: Opened local page read 2 (parent 0)
(00.024713)  65166: 	premap 0x00000000400000-0x00000000401000 -> 00007f16cf200000
(00.024733)  65166: 	premap 0x00000000401000-0x00000000402000 -> 00007f16cf201000
(00.024746)  65166: 	premap 0x00000000402000-0x00000000403000 -> 00007f16cf202000
(00.024764)  65166: 	premap 0x00000000403000-0x00000000404000 -> 00007f16cf203000
(00.024779)  65166: 	premap 0x00000000404000-0x00000000405000 -> 00007f16cf204000
(00.024806)  65166: 	premap 0x007f13d6e2c000-0x007f13d6e2f000 -> 00007f16cf205000
(00.024822)  65166: 	premap 0x007f13d6e2f000-0x007f13d6e57000 -> 00007f16cf208000
(00.024836)  65166: 	premap 0x007f13d6e57000-0x007f13d6fc4000 -> 00007f16cf230000
(00.024850)  65166: 	premap 0x007f13d6fc4000-0x007f13d7012000 -> 00007f16cf39d000
(00.024870)  65166: 	premap 0x007f13d7012000-0x007f13d7016000 -> 00007f16cf3eb000
(00.024890)  65166: 	premap 0x007f13d7016000-0x007f13d7018000 -> 00007f16cf3ef000
(00.024910)  65166: 	premap 0x007f13d7018000-0x007f13d7022000 -> 00007f16cf3f1000
(00.024929)  65166: 	premap 0x007f13d7034000-0x007f13d7038000 -> 00007f16cf3fb000
(00.024955)  65166: 	premap 0x007f13d7038000-0x007f13d703a000 -> 00007f16cf3ff000
(00.024969)  65166: 	premap 0x007f13d703a000-0x007f13d703b000 -> 00007f16cf401000
(00.024984)  65166: 	premap 0x007f13d703b000-0x007f13d7063000 -> 00007f16cf402000
(00.025005)  65166: 	premap 0x007f13d7063000-0x007f13d706d000 -> 00007f16cf42a000
(00.025030)  65166: 	premap 0x007f13d706d000-0x007f13d706f000 -> 00007f16cf434000
(00.025042)  65166: 	premap 0x007f13d706f000-0x007f13d7071000 -> 00007f16cf436000
(00.025070)  65166: 	premap 0x007ffcc683f000-0x007ffcc6860000 -> 00007f16cf438000
(00.025078)  65166: pr65166-2 Read 401000 1 pages
(00.025083)  65166: 	pr65166-2 Read page from self 401000/0
(00.025098)  65166: pr65166-2 Read 403000 1 pages
(00.025102)  65166: 	pr65166-2 Read page from self 403000/1000
(00.025109)  65166: pr65166-2 Read 404000 1 pages
(00.025113)  65166: 	pr65166-2 Read page from self 404000/2000
(00.025124)  65166: pr65166-2 Read 7f13d6e2c000 1 pages
(00.025135)  65166: 	pr65166-2 Read page from self 7f13d6e2c000/3000
(00.025146)  65166: pr65166-2 Read 7f13d6e2d000 1 pages
(00.025150)  65166: 	pr65166-2 Read page from self 7f13d6e2d000/4000
(00.025157)  65166: pr65166-2 Read 7f13d7012000 1 pages
(00.025161)  65166: 	pr65166-2 Read page from self 7f13d7012000/5000
(00.025167)  65166: pr65166-2 Read 7f13d7013000 1 pages
(00.025171)  65166: 	pr65166-2 Read page from self 7f13d7013000/6000
(00.025178)  65166: pr65166-2 Read 7f13d7014000 1 pages
(00.025182)  65166: 	pr65166-2 Read page from self 7f13d7014000/7000
(00.025189)  65166: pr65166-2 Read 7f13d7015000 1 pages
(00.025192)  65166: 	pr65166-2 Read page from self 7f13d7015000/8000
(00.025223)  65166: pr65166-2 Read 7f13d7016000 1 pages
(00.025227)  65166: 	pr65166-2 Read page from self 7f13d7016000/9000
(00.025235)  65166: pr65166-2 Read 7f13d7017000 1 pages
(00.025239)  65166: 	pr65166-2 Read page from self 7f13d7017000/a000
(00.025246)  65166: pr65166-2 Read 7f13d701c000 1 pages
(00.025250)  65166: 	pr65166-2 Read page from self 7f13d701c000/b000
(00.025260)  65166: pr65166-2 Read 7f13d701d000 1 pages
(00.025264)  65166: 	pr65166-2 Read page from self 7f13d701d000/c000
(00.025271)  65166: pr65166-2 Read 7f13d701e000 1 pages
(00.025275)  65166: 	pr65166-2 Read page from self 7f13d701e000/d000
(00.025281)  65166: pr65166-2 Read 7f13d701f000 1 pages
(00.025286)  65166: 	pr65166-2 Read page from self 7f13d701f000/e000
(00.025292)  65166: pr65166-2 Read 7f13d7020000 1 pages
(00.025296)  65166: 	pr65166-2 Read page from self 7f13d7020000/f000
(00.025303)  65166: pr65166-2 Read 7f13d7038000 1 pages
(00.025317)  65166: 	pr65166-2 Read page from self 7f13d7038000/10000
(00.025330)  65166: pr65166-2 Read 7f13d7039000 1 pages
(00.025334)  65166: 	pr65166-2 Read page from self 7f13d7039000/11000
(00.025340)  65166: pr65166-2 Read 7f13d706d000 1 pages
(00.025344)  65166: 	pr65166-2 Read page from self 7f13d706d000/12000
(00.025350)  65166: pr65166-2 Read 7f13d706e000 1 pages
(00.025353)  65166: 	pr65166-2 Read page from self 7f13d706e000/13000
(00.025359)  65166: pr65166-2 Read 7f13d706f000 1 pages
(00.025363)  65166: 	pr65166-2 Read page from self 7f13d706f000/14000
(00.025373)  65166: pr65166-2 Read 7f13d7070000 1 pages
(00.025376)  65166: 	pr65166-2 Read page from self 7f13d7070000/15000
(00.025382)  65166: pr65166-2 Read 7ffcc685c000 1 pages
(00.025386)  65166: 	pr65166-2 Read page from self 7ffcc685c000/16000
(00.025392)  65166: pr65166-2 Read 7ffcc685d000 1 pages
(00.025395)  65166: 	pr65166-2 Read page from self 7ffcc685d000/17000
(00.025406)  65166: pr65166-2 Read 7ffcc685e000 1 pages
(00.025409)  65166: 	pr65166-2 Read page from self 7ffcc685e000/18000
(00.025416)  65166: pr65166-2 Read 7ffcc685f000 1 pages
(00.025419)  65166: 	pr65166-2 Read page from self 7ffcc685f000/19000
(00.025435)  65166: nr_restored_pages: 5
(00.025439)  65166: nr_shared_pages:   21
(00.025442)  65166: nr_dropped_pages:  0
(00.025446)  65166: nr_enqueued:       0
(00.025449)  65166: nr_lazy:           0
(00.025453)  65166: Restore on-core sigactions for 65166
(00.025485)  65166: Restoring children in alien sessions:
(00.025490)  65166: Restoring children in our session:
(00.025518)  65166: Restoring 65166 to 65165 pgid
(00.025525)  65166: 	will call setpgid, mine pgid is 65613
(00.025561)  65165: Restoring resources
(00.025574)  65166: Restoring resources
(00.025588)  65165: Opening fdinfo-s
(00.025596)  65166: Opening fdinfo-s
(00.025597)  65165: tty: open driver pts id 0x4 index 4 (master 0 sid 63706 pgrp 65165 inherit 1)
(00.025604)  65166: 	Receive fd for 0
(00.025614)  65166: 	Receive fd for 1
(00.025618)  65166: 	Receive fd for 2
(00.025619)  65165: tty: Migrated slave peer 0x4 -> to fd 0
(00.025653)  65165: 		Create fd for 0
(00.025657)  65165: 			Going to dup 0 into 1
(00.025663)  65165: 			Going to dup 0 into 2
(00.025668)  65165: 		Send fd 0 to /crtools-fd-65166-effffffc0001004d
(00.025684)  65165: 		Send fd 0 to /crtools-fd-65166-effffffc0001004d
(00.025690)  65165: 		Send fd 0 to /crtools-fd-65166-effffffc0001004d
(00.025697)  65165: 	Receive fd for 1
(00.025701)  65165: 	Receive fd for 2
(00.026020)  65166: 		Create fd for 3
(00.026040)  65166: 	Receive fd for 0
(00.026052)  65166: Further fle=0x7f16d0381c38, pid=65166
(00.026056)  65166: 	Receive fd for 1
(00.026063)  65166: Further fle=0x7f16d0381c78, pid=65166
(00.026067)  65166: 	Receive fd for 2
(00.026073)  65166: Further fle=0x7f16d0381cb8, pid=65166
(00.026088)  65165: Error (criu/pidfd.c:177): pidfd: Could not wait on temporary process with pid: 65616: No child processes
(00.026094)  65165: Error (criu/pidfd.c:245): pidfd: Failed to delete dead_pidfd struct
(00.026101)  65165: Error (criu/pidfd.c:261): pidfd: Can't create pidfd 0x000006 NSpid: -1 flags: 0
(00.026126)  65165: Error (criu/files.c:1221): Unable to open fd=3 id=0x6
(00.026167)  65166: Restore via sigreturn
(00.026192) Error (criu/cr-restore.c:2314): Restoring FAILED.
(00.026789) Error (criu/cr-restore.c:1261): 65165 killed by signal 9: Killed
(00.027685) Error (criu/cgroup.c:1998): cg: cgroupd: recv req error: No such file or directory

@rst0git
Copy link
Member

rst0git commented Oct 18, 2024

@bsach64 Thank you for working on this!

Would it be possible to create a page describing how checkpoint/restore of pidfd works in https://criu.org/Category:Under_the_hood?

@bsach64
Copy link
Member Author

bsach64 commented Oct 18, 2024

@bsach64 Thank you for working on this!

Would it be possible to create a page describing how checkpoint/restore of pidfd works in https://criu.org/Category:Under_the_hood?

Sure @rst0git! I will do so in the near future!

bsach64 added a commit to bsach64/criu that referenced this issue Oct 28, 2024
@bsach64
pidfd: Allow processes to have common dead pidfd
9810c9e 
This patch ensures that the process that creates the tmp process is the
one that kills and waits for it when all pidfds have been opened.
We do this by keeping track of the count of dead pidfds that each
process has opened.
When the count for the creator of the tmp process reaches 0, it waits
for all other processes to open pidfds and then kills and waits for the
tmp process.

Fixes: checkpoint-restore#2496

Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
bsach64 added a commit to bsach64/criu that referenced this issue Oct 28, 2024
@bsach64
pidfd: Allow processes to have common dead pidfd
cb5d7ca 
This patch ensures that the process that creates the tmp process is the
one that kills and waits for it when all pidfds have been opened.
We do this by keeping track of the count of dead pidfds that each
process has opened.
When the count for the creator of the tmp process reaches 0, it waits
for all other processes to open pidfds and then kills and waits for the
tmp process.

Fixes: checkpoint-restore#2496

Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
@bsach64 bsach64 mentioned this issue Oct 28, 2024
Merged
bsach64 added a commit to bsach64/criu that referenced this issue Nov 6, 2024
@bsach64
pidfd: Allow processes to have common dead pidfd
dc63e6a 
This patch ensures that the process that creates the tmp process is the
one that kills and waits for it when all pidfds have been opened.
We do this by keeping track of the count of dead pidfds that each
process has opened.
When the count for the creator of the tmp process reaches 0, it waits
for all other processes to open pidfds and then kills and waits for the
tmp process.

Fixes: checkpoint-restore#2496

Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
bsach64 added a commit to bsach64/criu that referenced this issue Nov 7, 2024
@bsach64
pidfd: one process creates a helper and opens all fds to it
10d796e 
Currently, the `waitpid()` call on the tmp process can be made by a
process which is not its parent. This causes restore to fail.

This patch instead selects one process to create the tmp process and
open all the fds that point to it. These fds are sent to the correct
process(es).

Fixes: checkpoint-restore#2496

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
bsach64 added a commit to bsach64/criu that referenced this issue Nov 8, 2024
@bsach64
pidfd: one process creates a helper and opens all fds to it
b288b55 
Currently, the `waitpid()` call on the tmp process can be made by a
process which is not its parent. This causes restore to fail.

This patch instead selects one process to create the tmp process and
open all the fds that point to it. These fds are sent to the correct
process(es).

Fixes: checkpoint-restore#2496

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
bsach64 pushed a commit to bsach64/criu that referenced this issue Nov 8, 2024
@avagin @bsach64
pidfd: one process creates a helper and opens all fds to it
12d5bae 
Currently, the `waitpid()` call on the tmp process can be made by a
process which is not its parent. This causes restore to fail.

This patch instead selects one process to create the tmp process and
open all the fds that point to it. These fds are sent to the correct
process(es).

Fixes: checkpoint-restore#2496

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
mihalicyn pushed a commit to bsach64/criu that referenced this issue Nov 12, 2024
@avagin @mihalicyn
pidfd: one process creates a helper and opens all fds to it
40d6f82 
Currently, the `waitpid()` call on the tmp process can be made by a
process which is not its parent. This causes restore to fail.

This patch instead selects one process to create the tmp process and
open all the fds that point to it. These fds are sent to the correct
process(es).

Fixes: checkpoint-restore#2496

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
avagin added a commit that referenced this issue Nov 12, 2024
@avagin
pidfd: one process creates a helper and opens all fds to it
6f0ec7d 
Currently, the `waitpid()` call on the tmp process can be made by a
process which is not its parent. This causes restore to fail.

This patch instead selects one process to create the tmp process and
open all the fds that point to it. These fds are sent to the correct
process(es).

Fixes: #2496

Signed-off-by: Andrei Vagin <avagin@gmail.com>
Signed-off-by: Bhavik Sachdev <b.sachdev1904@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pidfd: tmp process is waited upon by correct process bsach64/criu
2 participants
@rst0git @bsach64