Skip to content

Latest commit

 

History

History
116 lines (93 loc) · 5.05 KB

workflow-executors.md

File metadata and controls

116 lines (93 loc) · 5.05 KB
Raw

Workflow Executors

A workflow executor is a process that conforms to a specific interface that allows Argo to perform certain actions like monitoring pod logs, collecting artifacts, managing container lifecycles, etc..

The executor to be used in your workflows can be changed in the configmap under the containerRuntimeExecutor key.

Docker (docker)

default

  • Reliability:
    • Most well-tested
    • Most popular
  • Least secure:
    • It requires privileged access to docker.sock of the host to be mounted which. Often rejected by Open Policy Agent (OPA) or your Pod Security Policy (PSP).
    • It can escape the privileges of the pod's service account
    • It cannot runAsNonRoot.
  • Equal most scalable:
    • It communicates directly with the local Docker daemon.
  • Artifacts:
    • Output artifacts can be located on the base layer (e.g. /tmp).
  • Configuration:
    • No additional configuration needed.

Kubelet (kubelet)

  • Reliability:
    • Second least well-tested
    • Second least popular
  • Secure
    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • runAsNonRoot - TBD, see #4186
  • Scalable:
    • Operations performed against the local Kubelet
  • Artifacts:
    • Output artifacts must be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
  • Configuration:
    • Additional Kubelet configuration maybe needed

Kubernetes API (k8sapi)

  • Reliability:
    • Well-tested
    • Popular
  • Most secure:
    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • Can runAsNonRoot
  • Least scalable:
    • Log retrieval and container operations performed against the remote Kubernetes API
  • Artifacts:
    • Output artifacts must be saved on volumes (e.g. emptyDir) and not the base image layer (e.g. /tmp)
  • Configuration:
    • No additional configuration needed.

Process Namespace Sharing (pns)

  • Reliability:
    • Well-tested
    • Popular
  • More secure:
    • No privileged access
    • cannot escape the privileges of the pod's service account
    • Can runAsNonRoot, if you use volumes (e.g. emptyDir) for your output artifacts
    • Processes are visible to other containers in the pod. This includes all information visible in /proc, such as passwords that were passed as arguments or environment variables. These are protected only by regular Unix permissions.
  • Scalable:
    • Most operations use local procfs.
    • Log retrieval uses the remote Kubernetes API
  • Artifacts:
    • Output artifacts can be located on the base layer (e.g. /tmp)
    • Cannot capture artifacts from a base layer which has a volume mounted under it
  • Configuration:
    • No additional configuration needed.
  • Process will no longer run with PID 1
  • Doesn't work for Windows containers.

https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/

Emissary (emissary)

This is the most fully featured executor.

This executor works very differently to the others. It mounts and empty-dir on all containers at /var/argo. The main container command is replaces by a new binary emissary which starts the original command in a sub-process and when it is finished, captures the outputs:

The init container creates these files:

  • /var/argo/emissary The emissary binary, copied from the argoexec image.
  • /var/argo/template A JSON encoding of the template.

In the main container, the emissary creates these files:

  • /var/argo/exitcode Will the sub-processes exit code (once process complete and all clean-up done).
  • /var/argo/outputs/parameters/${path} All output parameters are copied here, e.g. /tmp/message is moved to /var/argo/outputs/parameters/tmp/message`.
  • /var/argo/outputs/artifacts/${path}.tgz All output artifacts are copied here, e.g. /tmp/message is moved to /var/argo/outputs/artifacts/tmp/message.tgz`.
  • /var/argo/stderr A copy of stderr.
  • /var/argo/stdout A copy of stdout.

The wait container can create one file itself, used for terminating the sub-process.

  • /var/argo/signal The emissary binary listens to changes in this file, and signals the sub-process with the signal found in this file.

  • Reliability:

    • Least well-tested.
    • Least popular.

  • More secure:

    • No privileged access
    • Cannot escape the privileges of the pod's service account
    • Can runAsNonRoot.

  • Scalable:

    • It reads and writes to and from the container's disk and does not use any network APIs.

  • Artifacts:

    • Output artifacts can be located on the base layer (e.g. /tmp).

  • Configuration:

    • command must be specified for containers.