Add EiffelArtifactDeployedEvent?

[e-backmark-ericsson]

Description

Eiffel currently lacks some descriptiveness in the CD (Continuous Delivery/Deployment) area. One example is the possibility to connect a deployment of an artifact (Docker image for example) onto a staging or production environment. An idea would be to add an EiffelArtifactDeployedEvent (ArtD), that would link to ArtC using an ARTIFACT link. ArtD could contain fields to tie the artifact to a certain deployment environment and/or Continuous Delivery tool, such as Keptn/ArgoCD/Spinnaker (managing deployments) and/or Prometheus (managing monitoring) and/or Consul (managing service discovery).
Probably it would be better to use the event to say that a 'deployment process is started', linking to tools used for e.g. canary or blue/green deployments, than to say for each individual deployment that now this specific deployment instance is created. So, maybe EiffelArtifactDeploymentInitiatedEvent is a better name for it?

Motivation

To enable traceability using Eiffel events further into the Continuous Delivery/Deployment area

Exemplification

When e.g. ArgoCD starts to deploy a new Docker image onto a K8s cluster it could send the ArtD event linking to its own internal ArgoCD ids for the specific deployment process

Benefits

It would make it possible to connect and trace more parts of the CI/CD pipeline operations, namely the deployments onto production and/or staging environments.

Possible Drawbacks

This is a slightly new type of usage for Eiffel events, so maybe it could make it harder to distinguish the boundaries for what Eiffel events should be used for?

[k-hallen-ericsson]

Adding to Emils list of possible deployment environments/things that know about the deployment - Kubernetes liveness and readiness probes.

[e-backmark-ericsson]

Don't forget to compare this proposal with the existing Eiffel Operations Extension - https://github.com/Ericsson/eiffel-operations-extension, that already have an EiffelArtifactDeployedEvent defined.

[ebopalm]

Adding a comment to the reference of eiffel-operations-extensions. It was originally designed to add CD capabilities to Eiffel, to support production based and test/verification based delivery/deployment, which is very common to this issue. The potential difference might be the usage of the "service" notion compared to the suggested "artifact" notion. That difference might represent a much needed definition on how to (in Eiffel) represent various entities in software building blocks, ranging from smallest code-snippet to a deployed/running instance(s) of a small or large service-type product.

[e-backmark-ericsson]

@ebopalm , I'd say anything that is possible to relate to using a Purl could be an artifact. Be it a code snippet or a running instance/service - https://github.com/eiffel-community/eiffel/blob/master/eiffel-syntax-and-usage/glossary.md#artifact

[sodero]

A general comment; for us it makes a lot of sense to have an event for this. Further down the road we plan to extend the Eiffel pipeline to reach 'all the way' without resorting to something as generic as EiffelActivity*.

[d-stahl-ericsson]

I think there's a general consensus that this is useful. Who wants to take a stab at defining it?

[e-backmark-ericsson]

There is work ongoing in this area within CDF SIG Events right now. In order to not introduce anything in Eiffel that will not be compatible with the coming CDF Event protocol, we should probably wait for that work to progress a bit further, unless someone is really eager to get this event type in sooner.

[e-backmark-ericsson]

There is already an EiffelArtifactDeployedEvent supported in REMReM Semantics. I suspect it comes from the not-yet-published Eiffel Operations Extension. I expect no one is using that event type yet, so that we're free to define it in whatever way.

[magnusbaeck]

This was discussed at the May 2022 meetup (see notes) and at the 2022-06-09 community meeting. Here's my attempt to summarize (or paraphrase?) the outcome of the latter meeting:

• The new ServiceDefined event declares the existence of a particular "service", which could be anything for which "deployment" makes any sense (and is semantically different from an artifact that's just being published) but is typically a piece of executable software. Fields:
  • name: The name of the service. Not meant to be semantically significant.
  • endpointUris: One or more URIs exposed by the program, e.g. API endpoints. Optional.
  • uri: A URI with more information about the service. Optional.
• An EnvironmentDefined describes where a service runs. It could be an abstract concept like "sandbox" or "production" but it could also be something very concrete like a particular host.
• The new DeploymentDefined event announces that a composition of (typically) artifacts that describe the software binaries and the deployment configuration can be deployed. Fields:
  • uri: A URI that contains additional information about the deployment, such as its scope (e.g. a 5% canary). That's probably hard to encode into the events themselves.
• Activity events can be used to add information about the progress of the deployment. The ActT would typically have a CAUSE or a CONTEXT link to the DeploymentDefined event.
• A CLM can express that the deployment itself was successful, possibly with a CAUSE link to the deployment activity's ActF. The success criteria of a deployment isn't necessarily tied to the success of the main deployment activity (e.g. because you have stricter criteria than, say kubectl apply).

Additional thoughts:

• Not sure "service" is the best term.
• ServiceDefined only declares that a service has been defined, not that it's running.
• DeploymentDefined is used for both initial service spin-up, upgrades, and downgrades.
• DeploymentDefined is justified because it allows for pipeline actors that determine which compositions of artifacts are actually deployable, but maybe a CLM would do?
• We should probably consider having an event that expresses the turndown of a service.

cc:
@Isacholm
@t-persson

[m-linner-ericsson]

Thanks @magnusbaeck for adding the notes and creating the picture!

Linking:

• Wouldn't the deployment event link to a an Activity with a CONTEXT link? Wouldn't we send the deployment event during the activity?
• Do we want to link to the ActF event from a CLM? Its valid in the protocol today, but in my head we would rather use a CONTEXT link to a ActT (not giving the result you are out after but more in line with the protocol today)

When is it deployed?

• Reading on https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance when do we know when "something is successfully released to production"? Is that a reading CLM events? An analogy (maybe a bad one) is that we have an event for when something is published but here we don't and we don't use CLM event to signal an artifact published. In some sense a ServicePublished or ServiceDeployed would make it easier for an event listener to know when the service was finally deployed rather than listening to a bunch of CLM events.

Service:

• We flipped back and forth in the discussions but at some point we talked about that it was a Environment + Deployment = Service. Was that idea scrapped?
• If we instead would point the CLM event to the ServiceDefined, we could use that to talk about the confidence of the service. It would probably require to redraw the picture you have today.
• It would be good, as you state, to have an event expressing the turn-down of a service.

Clarity:

• For clarity would the CD event then point to the artifact being deployed e.g. the container image?
• Should we add some event between the ActS and ActF to tell what we intend to happen there. It's not too clear for me right now.

[magnusbaeck]

Good comments! I'll post what I have for now and continue later.

Wouldn't the deployment event link to a an Activity with a CONTEXT link? Wouldn't we send the deployment event during the activity?

Yes, that's another possibility. I imagined DeploymentDefined being used to define eligible deployments, i.e. there might be more DD events than what's actually picked up and deployed. The event could be sent outside of the deployment activity itself by a pipeline actor that monitors events and e.g. flags an artifact for deployment based on a CLM. However, sending DD as part of a deployment activity would also be fine, if that's when the (artifact, configuration, service) tuple is defined.

Do we want to link to the ActF event from a CLM? Its valid in the protocol today, but in my head we would rather use a CONTEXT link to a ActT (not giving the result you are out after but more in line with the protocol today)

I guess it depends on whether the CLM decision is made within that activity and what the success criteria of the deployment is. Is it enough that kubectl apply or ansible-playbook runs to completion without errors? Or that the service actually is alive, something not necessarily checked by the deployment activity.

... when do we know when "something is successfully released to production"? Is that a reading CLM events?

Yes, that's the idea.

An analogy (maybe a bad one) is that we have an event for when something is published but here we don't and we don't use CLM event to signal an artifact published.

Correct, but artifact publishing is also modeled at a higher, binary, level. We don't care when it starts, nor do we care about any logs produced during the act of publishing an artifact.

In some sense a ServicePublished or ServiceDeployed would make it easier for an event listener to know when the service was finally deployed rather than listening to a bunch of CLM events.

Not sure why there would be more CLM than there would be Service(Published|Deployed). But yeah, this is something we talked about during the meeting. Given the current proposal's DeploymentDefined my suggestion would be DeploymentFinished. I'm not sure what the point of having a dedicated event would be though (compared to reusing CLM), except that we don't have to establish a confidence level name that every uses (which might be a sufficient resaon).

We flipped back and forth in the discussions but at some point we talked about that it was a Environment + Deployment = Service. Was that idea scrapped?

I'm not sure how to translate you arithmetic expression to an event model :-) But are you saying that ServiceDeployed would be an event that links to an environment and a deployment that describes the artifact?

If we instead would point the CLM event to the ServiceDefined, we could use that to talk about the confidence of the service. It would probably require to redraw the picture you have today.

Hmm. It makes total sense to use CLM to express confidence in a particular service configuration. One important question is whether a service is a (relatively) static entity with a long lifetime or if we get a new service every time we make changes to it.

[magnusbaeck]

@e-backmark-ericsson and I talked about this at today's community meeting.

• The endpointUris field in the ServiceDefined event should probably be dropped. It's not clear how it should be used and whether this is the right place for it.
• ServiceDefined can have an optional PREVIOUS_VERSION link (or similar) to the ServiceDefined that represents the previous incarnation of the same service.
• We talked a lot about how to represent gradual rollouts (canaries etc). The current proposal includes that information in the DeploymentDefined (hidden in payload returned by the URI in the uri field) and assumes one DeploymentDefined and associated activity per rollout phase (one for 5% canary, one for 25% canary, ...). If desired you can have an activity that spans over all deployments until 100% is reached. Of course, in many cases a canary is implemented via feature flags or similar dynamic configurations that aren't necessarily captured with the CompositionDefined pointed to be the DeploymentDefined. It's not immediately clear how that would fit into the model.
• We briefly talked about how this would map to deployments to devices (e.g. mobile apps or firmware). Would every single device be represented by a set of (EnvironmentDefined, ServiceDefined, DeploymentDefined, ...) or would the deployment take place when the artifact is made available to the devices?
• We compared this model to the deployment events drafted for CDevents. One key difference is that CDevents defines a service as "a binary that is running" while we see it as a static entity. While the CDevents spec is far from done it would be wise to avoid such differences between Eiffel and CDevents to maximize the possibilities for translation and minimize the amount of confusion.

[magnusbaeck]

Answering the rest of @m-linner-ericsson's questions from last week:

For clarity would the CD event then point to the artifact being deployed e.g. the container image?

Yes. In many cases also the deployment configuration, if separate from the artifact itself (it usually is, in my experience).

Should we add some event between the ActS and ActF to tell what we intend to happen there. It's not too clear for me right now.

What kind of event would that be? I imagine that most deployment activities wouldn't produce any events themselves.

[m-linner-ericsson]

Some more thoughts that I hope can bring the discussion forward:

Reading from https://www.merriam-webster.com/dictionary/deployment and https://dictionary.cambridge.org/dictionary/english/deployment where they describe deployment as:

the use of something or someone in an effective way

the use of something or someone, especially in order to achieve a particular effect:

and

the act or movement of deploying or the state of being deployed:

an instance of use that involves something opening and spreading out

it sound to me that deployments are best modeled with activity events or a specialization of them.

On the other hand https://dictionary.cambridge.org/dictionary/english/service and https://www.merriam-webster.com/dictionary/service describes something more persistent

the work performed by one that serves

the occupation or function of serving

and

a government system or private organization that is responsible for a particular type of activity, or for providing a particular thing that people need:

a business that offers a particular type of help or work:

the act of dealing with customers in a shop, restaurant, or hotel by taking their orders, showing or selling them goods, etc.:

We could then have something like this

[e-backmark-ericsson]

Some complementing thoughts on top of Mattias comment above:

• A pipeline performs a deployment, in the same way it performs a build, or a test. I.e. a deployment is a specialized activity.
• The act of performing a build results in an artifact. A build activity aims at providing a built artifact.
• The act of performing a test results in a confidence level. A test activity aims at providing a confidence level on that artifact. The actual test steps performed in that activity could be dynamically defined, and gradually more and more complex and time consuming and resource consuming tests could be performed before the wanted confidence level is reached. There could be sub-confidence levels reached on the way to "full" confidence.
• The act of performing a deployment results in a service (or a service instance?). A deployment activity could be gradual, just as the test activity could. Similar to test activities the deployment activities could reach sub-levels of the full deployment, maybe called "deployment levels"? Should we introduce DeploymentLevelModified events, similar to CLM?

[m-linner-ericsson]

Regarding serverless: Can we consider a collection of serverless functions as a service? For a user of these functions they would see it as a service.

[magnusbaeck]

In the latest graph above it seems the "service" now describes the dynamic state, i.e. it includes the executing artifact, resulting in a ServiceDefined each time a deployment is made. That's probably okay if we have a PREVIOUS_XXX link to the previous incarnation of the service.

It's not clear why we need three new events for deployments instead of just using the regular activity events. We don't have to get to the bottom of this at this time but if we eventually choose that path it should be for good reasons. One such reason could be to be able to link to the ServiceDefined without using a CONTEXT or CAUSE link.

What if the service is defined as part of the deployment? It's highly likely that people are going to have CI jobs that send the CD that defines what's being deployed as well as the ServiceDefined. Or would that CI job be a separate activity that wraps the deployment activity?

[e-backmark-ericsson]

New suggestion, without the service concept:

[t-persson]

Here's an image drawn during the Eiffel Summit 2022.1