runtime: create state directory and structure

[crosbymichael]

For supporting things like hooks and updates we need a way to define runtime state of the container. This can be used to pass information to hooks so that they know the runtime state of the container and also used to dynamically make changes to resource allocations by running an update command for the runtime and have it reconcile the changes against the container's current state and the update requested.

For the hooks usecase the state should contain information such as:

• pid of the container's process
• paths to the namespaces (linux)
• paths to the cgroups setup for the container

For supporting updates the state should container information such as:

• pid of the container's process
• cgroup paths
• container root path
• cgroup/resource information
• device nodes
• rlimits
• sysctls
• mounts

Some of this information would be duplicated from the initial container's config so it maybe worth it to look into embedding the original config into the state structure.

[mrunalp]

I think it makes sense to embed the config into the state as is.

[LK4D4]

Also I think we should allow implementation to store state in any desired form. It can be registry key for windows for example. But hooks will always consume text version of state, so info about storage method will never leak outside implementation.

[philips]

What does "cgroup/resource information" mean?

[vmarmol]

/cc @vmarmol

[mrunalp]

@crosbymichael I think all the items in your lists make sense as state that could be passed to the hooks.

[wking]

On Tue, Jun 30, 2015 at 04:59:34PM -0700, Michael Crosby wrote:

Some of this information would be duplicated from the initial
container's config so it maybe worth it to look into embedding the
original config into the state structure.

I think embedding the initial config is less useful than embedding the
current effective config. For example, if you update the container to
mount a new directory, adjust limits, etc., the embedded config should
be a config capable of launching you directly into the current state
(modulo memory state in the running processes, which is what
checkpoint/restore is about).

[vmarmol]

We spoke about this in the meeting today. We don't think it makes sense to have here config where the source of truth is the kernel (e.g.: resource limits). We should have some immutable state which with some pre-defined operation can provide the current state. Otherwise keeping these in sync will be expensive and difficult to do.

[wking]

On Wed, Jul 22, 2015 at 09:58:45PM -0700, Victor Marmol wrote:

We don't think it makes sense to have here config where the source
of truth is the kernel (e.g.: resource limits).

The kernel is enforcing those resource limits, so why not make it the
source of truth?

We should have some immutable state…

Do you have an example workflow where a container's initial resource
limits (or whatever) matter and are different from the current
resource limits? I'm trying to understand the utility of a stale
state dump…

Otherwise keeping these in sync will be expensive and difficult to
do.

I'm not sure about difficult, but I'm not terribly worried about the
expense. These state dumps are just for hooks and occasional
host-initiated maintenance, right? I don't see us needing to refresh
them multiple times per second. Is there a particular check that you
expect to be difficult/expensive?

[philips]

@crosbymichael My concern with having device node, sysctl, rlmit, etc state parameters is that we are simply serializing Kernel features and state. Why not tell consumers to read from the source of truth instead?

[philips]

Gah, sorry for piling on I had this tab with a half finished response and the stuff from Victor hadn't loaded yet.

[crosbymichael]

Ya, in my PR for the state it is much more conservative and gives people just the information required to look in the correct locations.