Overlayfs and containers

Table of Contents Overlayfs nr. 1 user Containers wish list Why use overlayfs? Historical issues Recent progress Why are the features disabled? Container use cases Local instance Migratable instance Portable image Recommendations

Overlayfs nr. 1 user

Containers wish list

• shiftfs/fsid mapping
• Userns root mount (FS_USERNS_MOUNT) (coming up)
• "volatile" overlay (coming up)
• Containerized syncfs(2) (RFC)

Why use overlayfs?

• Because it is the default
• Share page cache
• Fast startup time
• Native diff - faster image build

Historical issues

• Non-standard behavior: inconsistent file data, inconsistent object identifier, broken hardlinks, failure to rename directory
• Userspace mitigations: yum-plugin-ovl, recursive mv

Recent progress

• Rename directories (redirect_dir=on) (v4.10)
• Preserve hardlinks (index=on) (v4.13)
• NFS export (nfs_export=on) (v4.16)
• Unique object identifier (xino=auto) (v4.12..v4.17)
• Consistent file data (excluding mmap) (v4.19)
• Metadata only copy up (metacopy=on) (v4.19)

Why are the features disabled?

• Backward compatibility with old kernels
• Enabling xino only impacts 32bit compat st_ino
• Impact some container use cases

Container use cases

Local instance

• Stays on one container host and storage driver (overlay2)
• Can safely enable: redirect_dir and metacopy
• Enabling index and nfs_export exposes mount leak bugs
• You've got nothing to lose!

Migratable instance

• Migrate instance between container hosts with same native storage driver (overlay2)
• Include trusted.overlay xattrs in image layers
• No need to convert opaque xattr and whiteout chardev to portable .wh. files
• Can safely enable: redirect_dir and metacopy
• The index and nfs_export features are NOT "natively migratable"

Portable image

• Enabling redirect_dir and metacopy requires more work
• vnd.oci.image.layer.v1 conforms to decade old overlay/aufs whiteouts and opaque directories
• Is it time for vnd.oci.image.layer.v2 with metacopy/redirect support?
• Downgrade to vnd.oci.image.layer.v1 format with overlayfs tools
  • IF is_metacopy_file_or_redirect_dir(upperdir/$path):
    • rm -rf imagedir/$path
    • cp -a mountdir/$path imagedir/$path

Recommendations

• Enable redirect_dir,metacopy for new instances
• Allow user to configure NFS exportable instances (index,nfs_export)
• Disable index before instance migration
• Downgrade redirect_dir,metacopy during image creation