Add selinux validate in runc exec #2031

lifubang · 2019-03-29T23:42:38Z

Signed-off-by: Lifubang lifubang@acmcoder.com

Fix regression introduced by cd96170 , the related commit is in opencontainers/selinux@e5c68ba#diff-08374585d1f5b66358d612f6292a3fae
There is no if processLabel == "" { check now.
Fixed by Fix SELinux failures on disabled SELinux Machines #2032

Fixes Container Creation error: Container creation error: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"write /proc/self/attr/keycreate: invalid argument\"" #2030 check nil for selinuxLabel before we label.SetKeyLabel, this is also introduced by cd96170 ;
Because the validate is :

runc/libcontainer/configs/validate/validator.go

Lines 96 to 107 in 84cba4c

    
           func (v *ConfigValidator) security(config *configs.Config) error { 
        
           	// restrict sys without mount namespace 
        
           	if (len(config.MaskPaths) > 0 || len(config.ReadonlyPaths) > 0) && 
        
           		!config.Namespaces.Contains(configs.NEWNS) { 
        
           		return fmt.Errorf("unable to restrict sys entries without a private MNT namespace") 
        
           	} 
        
           	if config.ProcessLabel != "" && !selinux.GetEnabled() { 
        
           		return fmt.Errorf("selinux label is specified in config, but selinux is disabled or not supported") 
        
           	} 
        
           	return nil 
        
           }

Fixed by Fix SELinux failures on disabled SELinux Machines #2032

Fix if we use selinuxLabel in runc exec, we should validate selinux again. If the system disable selinux, it will raise error exec failed: container_linux.go:345: starting container process caused "write /proc/self/attr/keycreate: invalid argument". For example:
(1) use '--process-label' in runc exec;
(2) use selinuxLabel in process.json when use --process in runc exec;
(3) add selinuxLabel to config.json in bundle dir after the container started.

thaJeztah · 2019-03-30T00:12:38Z

ping @rhatdan ptal

cyphar · 2019-03-30T02:00:27Z

libcontainer/setns_init_linux.go

+	// If config.json is modified after the container started, we should validate selinux again.
+	if l.config.ProcessLabel != "" && !selinux.GetEnabled() {
+		return fmt.Errorf("selinux label is specified in config, but selinux is disabled or not supported")
+	}


From memory, the config.json is not re-parsed for runc exec -- we use the config that's in /run/runc/$ctr/... which means that there's no chance it could change between runs (other than runc update but that only really does cgroup changes).

From memory, the config.json is not re-parsed for runc exec

No, it re-parsed in here:

runc/exec.go

Line 170 in 84cba4c

spec, err := loadSpec(specConfig)

@cyphar Besides re-parse config.json in bundle dir, there are 3 situations use selinux in runc exec, I have update the PR description.
(1) use '--process-label' in runc exec;
(2) add selinuxLabel in process.json when use --process in runc exec;
(3) add selinuxLabel to config.json in bundle dir after the container started.

I move the selinux validate to utils_linux.go when exec, so we can do this check before we start runc init. Does it make sense?

Yes, it would be much better to do it a lot earlier.

cyphar · 2019-03-30T02:01:01Z

libcontainer/setns_init_linux.go

-		if err := label.SetKeyLabel(l.config.ProcessLabel); err != nil {
-			return err
+		if l.config.ProcessLabel != "" {
+			if err := label.SetKeyLabel(l.config.ProcessLabel); err != nil {


@rhatdan I think that all of the Set*Label commands should be a no-op if l.config.ProcessLabel is "" -- so as to avoid these sorts of problems.

Yes, it does make sense for don't check nil in definition of label.Set*Label, because we maybe need to delete the process label, so we should check l.config.ProcessLabel is "" for all the Set*Label call in runc.

cyphar · 2019-03-30T02:05:14Z

Yeah, I didn't catch this when I reviewed #2012. Looks okay, minus my comments. We really should've fixed this for rc7... Guess it's time for an rc8 then...

rhatdan · 2019-03-30T10:40:25Z

Yes, I think we need to check this in Runc. What I don't understand is why this is blowing up, while the other calls do not.

But no SELinux calls should be made if the label is ""

rhatdan · 2019-03-30T10:42:20Z

LGTM

cyphar · 2019-03-31T04:00:34Z

What I don't understand is why this is blowing up, while the other calls do not.

I've figured it out -- it's because of older kernels and so this fix actually isn't sufficient (if you're using an older SELinux machine). I'm running AppArmor on a 5.0.3 kernel and I have a /proc/self/attr/keycreate file which will happily accept "" as an argument (even though it's wrong for SELinux code to be touching it on an AppArmor machine).

So, either runc or go-selinux need to detect whether /proc/self/attr/keycreate is present before trying anything (and maybe this should be the case for all writeCon cases). I would argue this would both be cleaner and make more sense in go-selinux than runc -- because then we'll have to be aware of SELinux internals.

And if we're going to do that sort of check, I think it would also make sense to no-op go-selinux calls where the label is "" -- unless "" actually makes sense as an SELinux label?

cyphar · 2019-03-31T06:02:41Z

I will submit a PR to go-selinux along the above lines, and then we can update this PR to re-vendor it and then remove some of the changes (but the "is SELinux enabled" for runc exec change should stay).

rhatdan · 2019-03-31T14:31:10Z

"" actuall makes sense as an SELinux label. This tells go-selinux to turn back on the default labeling.

label.SetProcessLabel("system_u:system_r:container_t:s0:c1,c2)
defer label.SetProcessLabel("")

cyphar · 2019-03-31T14:56:14Z

Hmm, so that means really runc should be gating all Set.*Label calls with selinux.IsEnabled (or the standard Label != "")? We do this already, but not nearly as explicitly as it appears we should be -- because it's a bit weird for a SELinux library to write to /proc/self/attr even though SELinux isn't in use. It would end up with weird results for AppArmor profiles (for instance).

rhatdan · 2019-03-31T15:02:50Z

We could embed the isselinuxenabled check in the label calls, and only check it if the "" is passed.

cyphar · 2019-03-31T15:17:42Z

I will send a PR with these changes (the is-selinux-enabled checks and the backwards-compatibility changes) tomorrow.

lifubang · 2019-04-02T00:55:10Z

Dear experts, after I see opencontainers/selinux#49 , do we really need patch 49 in runc? Because consider:

label.SetProcessLabel(labelVar)
defer label.SetProcessLabel("")

If labelVar is not empty, it is checked whether selinux enabled or not by runc.
If labelVar is empty, the code becomes:

label.SetProcessLabel("")
defer label.SetProcessLabel("")

I think it has no meanings to the system even though selinux is enabled.

So I think we should check labelVal in runc, just like check config.cwd is empty or not.

If I'm wrong, please point out. Thanks.

lifubang · 2019-04-02T01:03:11Z

If labelVar is empty, I think this means the user doesn't want to use selinux in runc.
We shouldn't call any selinux API at that time.

cyphar · 2019-04-02T01:13:51Z

@lifubang While it would make the operations a no-op, it's much better to do it in go-selinux so that we don't forget to do it in runc (like we forgot in #2012 and in several other places throughout runc). Function calls are cheaper than future maintenance work when someone forgets to add a check again.

lifubang · 2019-04-02T01:20:24Z

Function calls are cheaper than future maintenance work when someone forgets to add a check again.

I know what's the purpose now. Thank you for your explanation.

rhatdan · 2019-04-02T13:38:45Z

I have opened this PR opencontainers/selinux#49 to fix the issue in go-selinux.

lifubang · 2019-04-03T00:02:23Z

As the regression is fixed by #2032 , so this PR becomes enhancement. The main purpose of this PR is:

If the user don't provide selinuxLabel, I think this means the user doesn't want to use selinux in runc, so we shouldn't call any selinux API at that time.
Add selinux validate if the user provide selinuxLabel when runc exec.

cyphar · 2019-04-03T03:53:22Z

libcontainer/setns_init_linux.go

@@ -34,10 +34,12 @@ func (l *linuxSetnsInit) Init() error {
 	defer runtime.UnlockOSThread()

 	if !l.config.Config.NoNewKeyring {
-		if err := label.SetKeyLabel(l.config.ProcessLabel); err != nil {
-			return err
+		if l.config.ProcessLabel != "" {


These checks aren't necessary anymore.

cyphar · 2019-04-03T03:54:16Z

My only comment is we don't need any of the ProcessLabel != "" checks. Everything else looks fine. I'll send out the vote for an rc8 release once this is merged.

Signed-off-by: lifubang <lifubang@acmcoder.com>

cyphar · 2019-04-03T04:15:10Z

LGTM. Thanks @lifubang.

crosbymichael · 2019-04-03T19:51:27Z

LGTM

no changes in vendored files full diff: opencontainers/runc@v1.0.0-rc7...029124d - opencontainers/runc#2031 Add selinux validate in runc exec - opencontainers/runc#2032 Fix SELinux failures on disabled SELinux Machines - addresses opencontainers#2030 "container init caused "write /proc/self/attr/keycreate: invalid argument" Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

… stretch-backport v1.0.0~rc8 This is a hot-fix for v1.0.0-rc7, and fixes a regression on old kernels (which don't support keycreate labeling). Users are strongly encouraged to update, as this regression was introduced in 1.0.0-rc7 and has blocked many users from updating to mitigate CVE-2019-5736. Bugs: opencontainers#2032 opencontainers#2031 opencontainers#2043 At the moment the only outlying issue before we can release 1.0.0 is some spec discussions we are having about OCI hooks and how to handle the integration with existing NVIDIA hooks. We will do our best to finish this work as soon as we can. Thanks to the following people who made this release possible: * Aleksa Sarai <asarai@suse.de> * Daniel J Walsh <dwalsh@redhat.com> * lifubang <lifubang@acmcoder.com> * Michael Crosby <crosbymichael@gmail.com> * Mrunal Patel <mrunal@me.com> Vote: +4 -0 opencontainers#1 Signed-off-by: Aleksa Sarai <asarai@suse.de>

cyphar reviewed Mar 30, 2019

View reviewed changes

thaJeztah mentioned this pull request Mar 30, 2019

update runc to v1.0.0-rc7 moby/moby#38969

Closed

lifubang force-pushed the selinux branch 2 times, most recently from 8d32762 to dda6158 Compare March 31, 2019 02:26

cyphar mentioned this pull request Mar 31, 2019

Container Creation error: Container creation error: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"write /proc/self/attr/keycreate: invalid argument\"" #2030

Closed

amimof mentioned this pull request Apr 2, 2019

Update component versions amimof/kubernetes-the-right-way#44

Merged

lifubang changed the title ~~Fix several selinux issues~~ Add selinux validate in runc exec Apr 3, 2019

cyphar reviewed Apr 3, 2019

View reviewed changes

lifubang force-pushed the selinux branch from dda6158 to b1ea070 Compare April 3, 2019 04:00

add selinux label for runc exec

3e6688f

Signed-off-by: lifubang <lifubang@acmcoder.com>

lifubang force-pushed the selinux branch from b1ea070 to 3e6688f Compare April 3, 2019 04:09

crosbymichael merged commit 029124d into opencontainers:master Apr 3, 2019

schu mentioned this pull request Apr 4, 2019

Update to runc v1.0.0-rc8 once available schu/kubedee#5

Closed

This was referenced Apr 5, 2019

[release/1.2] bump runc to 029124da (v1.0.0-rc7-6-g029124da) containerd/containerd#3183

Merged

[release/1.1] bump runc to 029124da (v1.0.0-rc7-6-g029124da) containerd/containerd#3184

Merged

tomalok mentioned this pull request Apr 6, 2019

community/runc: patch for containerd-1.2.6 alpinelinux/aports#6965

Closed

AkihiroSuda mentioned this pull request Apr 22, 2019

Question: runc 1.0.0 release plan #1905

Closed

lifubang deleted the selinux branch May 28, 2020 01:36

brandond mentioned this pull request Apr 6, 2021

Unable to build cluster with CIS profile (cis-1.5) enabled rancher/rke2#851

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add selinux validate in runc exec #2031

Add selinux validate in runc exec #2031

lifubang commented Mar 29, 2019 •

edited

Loading

thaJeztah commented Mar 30, 2019

cyphar Mar 30, 2019

lifubang Mar 30, 2019

lifubang Mar 30, 2019 •

edited

Loading

lifubang Mar 31, 2019

cyphar Mar 31, 2019

cyphar Mar 30, 2019

lifubang Mar 30, 2019

cyphar commented Mar 30, 2019

rhatdan commented Mar 30, 2019

rhatdan commented Mar 30, 2019

cyphar commented Mar 31, 2019 •

edited

Loading

cyphar commented Mar 31, 2019 •

edited

Loading

rhatdan commented Mar 31, 2019

cyphar commented Mar 31, 2019

rhatdan commented Mar 31, 2019

cyphar commented Mar 31, 2019

lifubang commented Apr 2, 2019

lifubang commented Apr 2, 2019 •

edited

Loading

cyphar commented Apr 2, 2019

lifubang commented Apr 2, 2019

rhatdan commented Apr 2, 2019

lifubang commented Apr 3, 2019

cyphar Apr 3, 2019

lifubang Apr 3, 2019

cyphar commented Apr 3, 2019

cyphar commented Apr 3, 2019 •

edited by caniszczyk

Loading

crosbymichael commented Apr 3, 2019 •

edited by caniszczyk

Loading

	func (v ConfigValidator) security(config configs.Config) error {
	// restrict sys without mount namespace
	if (len(config.MaskPaths) > 0 \|\| len(config.ReadonlyPaths) > 0) &&
	!config.Namespaces.Contains(configs.NEWNS) {
	return fmt.Errorf("unable to restrict sys entries without a private MNT namespace")
	}
	if config.ProcessLabel != "" && !selinux.GetEnabled() {
	return fmt.Errorf("selinux label is specified in config, but selinux is disabled or not supported")
	}

	return nil
	}

Add selinux validate in runc exec #2031

Add selinux validate in runc exec #2031

Conversation

lifubang commented Mar 29, 2019 • edited Loading

thaJeztah commented Mar 30, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lifubang Mar 30, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyphar commented Mar 30, 2019

rhatdan commented Mar 30, 2019

rhatdan commented Mar 30, 2019

cyphar commented Mar 31, 2019 • edited Loading

cyphar commented Mar 31, 2019 • edited Loading

rhatdan commented Mar 31, 2019

cyphar commented Mar 31, 2019

rhatdan commented Mar 31, 2019

cyphar commented Mar 31, 2019

lifubang commented Apr 2, 2019

lifubang commented Apr 2, 2019 • edited Loading

cyphar commented Apr 2, 2019

lifubang commented Apr 2, 2019

rhatdan commented Apr 2, 2019

lifubang commented Apr 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cyphar commented Apr 3, 2019

cyphar commented Apr 3, 2019 • edited by caniszczyk Loading

crosbymichael commented Apr 3, 2019 • edited by caniszczyk Loading

lifubang commented Mar 29, 2019 •

edited

Loading

lifubang Mar 30, 2019 •

edited

Loading

cyphar commented Mar 31, 2019 •

edited

Loading

cyphar commented Mar 31, 2019 •

edited

Loading

lifubang commented Apr 2, 2019 •

edited

Loading

cyphar commented Apr 3, 2019 •

edited by caniszczyk

Loading

crosbymichael commented Apr 3, 2019 •

edited by caniszczyk

Loading