zpkg: a package manager for zos

[muhamadazmy]

Right now the base image include all the base binaries and libraries to boot the node + some basic runtime packages (binaries that are need for zos operation).

During boot the node still also install some more packages in the form of flist. The node blindly copy all the flists content from a specific repo on the hub. Those package will contain binaries and libraries. the compatibility of the libraries are not granted (except by being always careful while building) to make sure we don't override one library (thing libc) with an older/newer version that might cause other binaries to fail.

This is why we usually prefer static builds for this kind of "packages" but it's not always possible.

What I think is instead, we need to introduce a new pkg manager that integrates well with the current system.
So basically it needs to:

• Use flist (packages) to install binaries
• Support dependency
  • A package X can depend on lib package Y
• Consistent build mechanism for all packages. The zos build system for runtime package need to integrate with the package building mechanism so it
  • Base image libraries must all come from the flists (packages) otherwise we can endup with the situation as explained below
  • Common libraries need to come from a single package so multiple binaries can just depend on this "flist" package without reincluding the same libraries

This sound (and is) pretty basic package management to make sure build is always consistent and never break an installed package by a broken flist build. We should consider looking around first for existing package management system and if we can reuse them to work with flists before we jump into building our own solution.

Bad Scenario 1 that makes this necessary

Right now github is deprecating the support for ubuntu 18.04. This is the one that we use to build zos base image. Hence all base libs are from ubuntu (well, the ones that we don't rebuild ourselves - like glibc). So we need to update this to use latest ubuntu.

This means that newer images has newer libs, so newly booted nodes will have newer libs, but the older ones (that are already in the field) won't.

Now we need to rebuild the new "pakcages" to depend on the same glibc version (and other libs) to work on the new images. this means it will break for older nodes ... well without package management

[muhamadazmy]

If we approve on building the package manager I can start specing it into details. The main goal is

• Smaller zos image it can only have (zinit, internet service, and the pkg manager) (in theory that's all what is needed but we might need more, like busybox or other components)
• The pkg manager start by updating itself, then installing the main umbrella package flist
• the umbrella package has dependency on the main zos components, which has dependency on other libs,
• eventually the image will pull in everything that is consistent because the dependency has correct version.
• the builder when building all utils, will always use the right versions so all flists should be consistent
• we can safely test new builds versions on test VMs without affecting the environments

[muhamadazmy]

Initial requirements

zpkg uses a db (sqlite) to track

• installed packages

  • id ) generated by system
  • name ) full package name 'repo/name'
  • version ) from metadata, must match name ?

• dependencies

  • pkg id
  • dep id

• files

  • pkg id
  • abs file path

Install package(s)

Phase 1

• build dependency tree.
  (with order)
• dependency tree must not have 2 versions of the same package. otherwise the tree is not consistent and not installable.
• for each package in the tree, a consistency check is done against the installed packages. To make sure that installing this package will not break already installed packages (if it's a dependency by a package, this package then must be part of the installation tree).
• for each file in each package, files are checked against installed files (also files from the same installation tree). Conflicts are detected, the installation is considered inconsistent if conflicting files found that don't belong to the same package. Un-tracked files are not considered conflict hence are okay to override.

Phase 2

• once phase 1 passes, phase 2 is basically extracting the files.
• update the database with each update (file extracted, etc...)

Hooks

Since using this package manager will phase out the current update procedure (currently maintained by identityd). but one of the steps taken by identity is to stop, restart and wait for services. This now need to to be done by the pkg manager. hence a package need to be able to define hooks that are understood by the pkg manager for example

.pkg/
   hooks/
      pre-install
      post-install

the hooks need to have a clear definitions. They can be any executable file (script or binary) and run at defined states.

Example: ZOS package can have a stop all pre-install hook. Another start all hook as a post install.

Required changes

Base image

0-initramfs (the zos base image) can be minimized to only (theoretically) have only the:

• kernel (and modules)
• zinit
• internet (from zos which initialized system networking)
• zpks (the package manager)

other packages that can also be part of the base image is busybox and openssh

on boot, zinit is configured (with proper yaml) to kickstart zpkg with proper installation command to install zos binaries. This will pull everything eventually if the dependencies tree is complete.

ZOS

zos currently maintain a list of all system packages that are installed during runtime. those need to be extended (and properly moved to another dedicated repo, that will include all pkgs from both 0-initramfs and zos.

This new repo on build can build all packages (properly so each contain only it's binaries/libs) and build proper pkg metadata (like versions and dependencies).

The builder repo can build a package from source, or by packing the binaries directly from the system. making sure it only include it's own files not files from a dependency.

The builder repo should be smart enough (or has the options) to specify what packages need repackaging. Will probably need a consistency check after all builds are completely. may be by means of using zpkg itself to check packages consistency. While zpkg on the nodes should detect inconsistency there is no need to break the public repo. So it's better to only push builds that produce a consistent installation tree.

issues

• What if failure happens during phase 2 (failed to download files or ran out of space)?
• Make sure pkg db (suggest sqlx since that's already used by rfs metadata) is locked during the operation to prevent other instance of the package manager to work on the same db.

[muhamadazmy]

Deprecated in favor of #2012