WIP - Argocd/Flux Comparison Document #47
Conversation
Signed-off-by: Christian Hernandez <chernand@redhat.com>
Signed-off-by: Christian Hernandez <chernand@redhat.com>
Signed-off-by: Christian Hernandez <chernand@redhat.com>
christianh814
force-pushed
the
argocd-flux-comparison
branch
from
bab2215 to
75fe9b4
Compare
|
Another note, we started refactoring this into a new structure, to make clear to end users both the technical and philosophical similarities and differences. That's currently in this hackmd doc, but we intend to fold these together as another commit in this PR. |
There was a problem hiding this comment.
I think this is a great start. From what I understand, this is meant to be a GitOps focused view of the two products? Or is it more holistic than that?
I have left some inline comments regarding what I think could be added or clarified.
Also I think whatever tool you used to copy this Markdown has done some weird spacing with your links in some places, and inserted double spaces as well. Probably worth combing over everything to fix those as a nit pick.
|
|
||
| ## Argo CD | ||
|
|
||
| Argo CD reconciliation is a global setting. This is one setting that can be modified via the [Argo CD configuration ConfigMap](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/argocd-cm.yaml#L240). By default, this is set to 3 minutes and can be changed by the end user. Although you have a choice of disabling reconciliation globally or per-application, you can only set the reconciliation time globally. This means you cannot set the reconciliation time differently for each [Application](https://argo-cd.readthedocs.io/en/stable/core_concepts/). |
There was a problem hiding this comment.
| Argo CD reconciliation is a global setting. This is one setting that can be modified via the [Argo CD configuration ConfigMap](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/argocd-cm.yaml#L240). By default, this is set to 3 minutes and can be changed by the end user. Although you have a choice of disabling reconciliation globally or per-application, you can only set the reconciliation time globally. This means you cannot set the reconciliation time differently for each [Application](https://argo-cd.readthedocs.io/en/stable/core_concepts/). | |
| Argo CD reconciliation is a global setting. This is one setting that can be modified via the [Argo CD configuration ConfigMap](https://github.com/argoproj/argo-cd/blob/891d9b2f380e1879474ab57c1adf21f5d1678759/docs/operator-manual/argocd-cm.yaml#L246). By default, this is set to 3 minutes and can be changed by the end user. Although you have a choice of disabling reconciliation globally or per-application, you can only set the reconciliation time globally. This means you cannot set the reconciliation time differently for each [Application](https://argo-cd.readthedocs.io/en/stable/core_concepts/). |
There was a problem hiding this comment.
Argo definitely hand slaps you for using k apply -f manually and sets it back to the last declared state in the Git repo that's sync'ed
|
|
||
| Flux uses Kustomize heavily. Its[ Kustomize Controller](https://fluxcd.io/docs/components/kustomize/) renders and applies all of the manifests that Flux syncs from the source repository, even when using Raw Kubernetes YAML. Flux will create a `kustommization.yaml` file “on the fly”, if Kustomize wasn’t detected. | ||
|
|
||
| When you first bootstrap Flux onto your cluster, it does so by using a Kustomized application called “flux-system” that serves as an entry point for deploying other resources. This entrypoint does a sort of an “autodiscovery” of other YAMLs by doing a Kustomize recursive search in the directory (this is on by default). All someone has to do is create a directory with some YAML, and Flux will “auto detect” it and apply it to the cluster. However, there is no builtin mechanism for ordering. The end user must wait for subsequent reconciliation loops to get a successful application. |
There was a problem hiding this comment.
Flux also supports decrypting secrets, adding merge patches and overriding image names as part of its configuration.
|
|
||
| <span style="text-decoration:underline;">Raw Kubernetes YAML:</span> | ||
|
|
||
| Argo CD also supports Kustomize and raw Kubernetes YAML. It can, however, apply raw YAML files directly, without using Kustomize. This can have side effects of Argo CD Applications not syncing right aways as resources need to be reapplied in a retry ([which can be set up per Application](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/application.yaml#L111-L116)), or use ordering built into Argo CD. Argo CD does the equivalent of `kubectl apply -f` or a `kustomzie build` if Kustomize is detected. This is by design as the Argo CD creators wanted to emulate the `kubectl` client as closely as possible. |
There was a problem hiding this comment.
Does this properly use server side applies, though?
There was a problem hiding this comment.
Should this fall under "Break Glass" potentially?
|
|
||
| <span style="text-decoration:underline;">Helm:</span> | ||
|
|
||
| Argo CD also has support for Helm, but approaches it in a different way. Argo CD does not use the Helm golang library directly, but instead, uses helm to render the template files and pipes them directly to `kubectl`. Helm chart hooks are converted into Argo CD Syncwaves/Hooks (more on this next) where it can.This has the side effect of users not being able to use the `helm` cli against the cluster because `helm install` wasn’t called in the first place (i.e. `helm ls `will return empty). |
There was a problem hiding this comment.
Helm chart hooks are converted into Argo CD Syncwaves/Hooks (more on this next) where it can
This sentence wasn't finished
|
|
||
| ## FluxCD | ||
|
|
||
| FluxCD provides a guide for managing encrypted secrets with [Mozilla SOPS](https://fluxcd.io/docs/guides/mozilla-sops/) or [Sealed Secrets](https://fluxcd.io/docs/guides/sealed-secrets/) so that Kubernetes secrets can be stored safely in Git, secured by any of OpenPGP, AWS KMS, GCP KMS and Azure Key Vault. Flux’s Kustomization objects natively support decrypting Mozilla SOPS secrets. |
There was a problem hiding this comment.
I'm not sure if this is worth calling out specifically for Flux? The documentation provides guides for these methods, but Flux doesn't provide any additional value on top of these.
There was a problem hiding this comment.
Yes. We are. And we typically point to a part of the CNCF Landscape (sealed secrets, Vault, etc.).
|
|
||
| # Image updates automation | ||
|
|
||
| Image updating is the process of notifying the GitOps controller that a new image has been created and pushed into a repository. This helps in expediting the process of rolling out new versions of applications. |
There was a problem hiding this comment.
Is it worth calling out that this feature isn't directly tied into GitOps? I think it's great that it's being explored, but it's more related to CD.
|
|
||
| ## Argo CD | ||
|
|
||
| Argo CD uses a separate controller called the ["Argo CD Image Updater"](https://argocd-image-updater.readthedocs.io/en/stable/), which takes a similar approach of “writing back” image updates to the YAML manifest in the git repository. The image updater will do this in the form of a[ Kustomize image update](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/image.md). Currently, the tool will only work if you’re using Kustomize or Helm, not not raw (non-kustomized) YAML files. |
There was a problem hiding this comment.
| Argo CD uses a separate controller called the ["Argo CD Image Updater"](https://argocd-image-updater.readthedocs.io/en/stable/), which takes a similar approach of “writing back” image updates to the YAML manifest in the git repository. The image updater will do this in the form of a[ Kustomize image update](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/image.md). Currently, the tool will only work if you’re using Kustomize or Helm, not not raw (non-kustomized) YAML files. | |
| Argo CD uses a separate controller called the ["Argo CD Image Updater"](https://argocd-image-updater.readthedocs.io/en/stable/), which takes a similar approach of “writing back” image updates to the YAML manifest in the git repository. The image updater will do this in the form of a[ Kustomize image update](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/image.md). Currently, the tool will only work if you’re using Kustomize or Helm and not raw (non-kustomized) YAML files. |
|
|
||
| # RBAC | ||
|
|
||
| With RBAC, each tool not only takes a different technical approach, but also a different philosophical approach. \ |
There was a problem hiding this comment.
| With RBAC, each tool not only takes a different technical approach, but also a different philosophical approach. \ | |
| With RBAC, each tool not only takes a different technical approach, but also a different philosophical approach. |
|
|
||
| ## Argo CD | ||
|
|
||
| Argo CD has an[ ACL based RBAC](https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/), independent of Kubernetes. Argo CD also has a separate group/user mechanism, which is also independent of Kuberenetes. Normally, a user sets up Argo CD’s **_ServiceAccount_** with full admin, and manages users using Argo CD’s user/group mechanism. Or a user can use a combination of the two to get the desired outcome. |
There was a problem hiding this comment.
| Argo CD has an[ ACL based RBAC](https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/), independent of Kubernetes. Argo CD also has a separate group/user mechanism, which is also independent of Kuberenetes. Normally, a user sets up Argo CD’s **_ServiceAccount_** with full admin, and manages users using Argo CD’s user/group mechanism. Or a user can use a combination of the two to get the desired outcome. | |
| Argo CD has an [ACL based RBAC](https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/), independent of Kubernetes. Argo CD also has a separate group/user mechanism, which is also independent of Kubernetes. Normally, a user sets up Argo CD’s **_ServiceAccount_** with full admin, and manages users using Argo CD’s user/group mechanism. Or a user can use a combination of the two to get the desired outcome. |
|
|
||
| ## FluxCD | ||
|
|
||
| FluxCD has the concept of connecting to the remote clusters via Kustomization’s (and HelmRelease’s) _KubeConfig_ setting. FluxCD uses a “pull-based” design, where there needs to be an instance of FluxCD running inside of each cluster. A centralized “management cluster” is another possible configuration, where one Flux instance manages workloads on many clusters. “Multi Cluster '' is achieved through [design and process](https://github.com/fluxcd/flux2-multi-tenancy), via a reference architecture. [Other tools, like CAPI, need to be brought in](https://www.youtube.com/watch?v=7W27tAv7Tvs), to install and manage the clusters. |
There was a problem hiding this comment.
| FluxCD has the concept of connecting to the remote clusters via Kustomization’s (and HelmRelease’s) _KubeConfig_ setting. FluxCD uses a “pull-based” design, where there needs to be an instance of FluxCD running inside of each cluster. A centralized “management cluster” is another possible configuration, where one Flux instance manages workloads on many clusters. “Multi Cluster '' is achieved through [design and process](https://github.com/fluxcd/flux2-multi-tenancy), via a reference architecture. [Other tools, like CAPI, need to be brought in](https://www.youtube.com/watch?v=7W27tAv7Tvs), to install and manage the clusters. | |
| FluxCD has the concept of connecting to the remote clusters via Kustomization’s (and HelmRelease’s) _KubeConfig_ setting. FluxCD uses a “pull-based” design, where there needs to be an instance of FluxCD running inside of each cluster. A centralized “management cluster” is another possible configuration, where one Flux instance manages workloads on many clusters. “Multi Cluster" is achieved through [design and process](https://github.com/fluxcd/flux2-multi-tenancy), via a reference architecture. [Other tools, like CAPI, need to be brought in](https://www.youtube.com/watch?v=7W27tAv7Tvs), to install and manage the clusters. |
| @@ -0,0 +1,275 @@ | |||
| # Continuous Deployment on Kubernetes: What’s the Difference Between Argo CD and FluxCD | |||
There was a problem hiding this comment.
Thank you for following through on my rather terse comment 🙏
(Context: Flux's github org and website domain use "fluxcd" ("flux" was gone already), which has caused ongoing confusion, even amongst its maintainers...)
|
|
||
| If you’ve heard of GitOps and its benefits, you may have come across two open source projects in the Cloud Native Computing Foundation: Argo CD and FluxCD. Members of these projects are involved in the CNCF’s OpenGitOps initiative, and both are incubating projects. With GitOps, you increase reliability and velocity, lower downtime, and can enjoy other benefits through key GitOps capabilities such as reconciliation. | ||
|
|
||
| When building out a Kubernetes cluster, the actual pipelines for the code behind applications is just as important as the firewalls, load balancers and provisioning systems at the front. There are many ways to build out application CI/CD pipelines in Kubernetes, but in this article, we are going to focus very specifically on two options for Continuous Deployment: [Flux CD](https://fluxcd.io/) (specifically Flux v2) and [Argo CD](https://argoproj.github.io/cd/). |
There was a problem hiding this comment.
The most common reference is CI/CD not CI & CD.
| @@ -0,0 +1,275 @@ | |||
| # Continuous Deployment on Kubernetes: What’s the Difference Between Argo CD and FluxCD | |||
There was a problem hiding this comment.
In the context of:
(Proposed) GitOps Principle 5. Operated in a closed loop
Software agents observe desired state and meta-state to take actions based on policy when the desired state cannot be reached. This may include things like notifications, rollbacks, etc.
There was a problem hiding this comment.
In the context of:
GitOps Principle 4. Continuously Reconciled
Software agents continuously observe actual system state and attempt to apply the desired state.
|
|
||
| If you’ve heard of GitOps and its benefits, you may have come across two open source projects in the Cloud Native Computing Foundation: Argo CD and FluxCD. Members of these projects are involved in the CNCF’s OpenGitOps initiative, and both are incubating projects. With GitOps, you increase reliability and velocity, lower downtime, and can enjoy other benefits through key GitOps capabilities such as reconciliation. | ||
|
|
||
| When building out a Kubernetes cluster, the actual pipelines for the code behind applications is just as important as the firewalls, load balancers and provisioning systems at the front. There are many ways to build out application CI/CD pipelines in Kubernetes, but in this article, we are going to focus very specifically on two options for Continuous Deployment: [Flux CD](https://fluxcd.io/) (specifically Flux v2) and [Argo CD](https://argoproj.github.io/cd/). |
There was a problem hiding this comment.
The most common reference is CI/CD not CI & CD.
|
I just want to say that using GitHub for mass edits on a doc like this is quite the inconvenience. I cannot follow the conversation at all(why are there videos now???). If this were something I was taking the lead on, I'd close this PR, have a conversation with stakeholders about how the perfect can't be the enemy of the good, go back to a Google Doc and ask folks to be very pithy with their comments. If anyone feels whole sections of the doc are missing, it shouldn't be in markdown in GitHub. The recommendation system in GitHub is good for code, terrible for prose. I'm completely lost. |
When we've reached that feeling in the past, we'd pause the PR and move to a more real-time doc like Google docs, or when markdown like this, to HackMD. |
|
|
||
| <span style="text-decoration:underline;">Raw Kubernetes YAML:</span> | ||
|
|
||
| Argo CD also supports Kustomize and raw Kubernetes YAML. It can, however, apply raw YAML files directly, without using Kustomize. This can have side effects of Argo CD Applications not syncing right aways as resources need to be reapplied in a retry ([which can be set up per Application](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/application.yaml#L111-L116)), or use ordering built into Argo CD. Argo CD does the equivalent of `kubectl apply -f` or a `kustomzie build` if Kustomize is detected. This is by design as the Argo CD creators wanted to emulate the `kubectl` client as closely as possible. |
There was a problem hiding this comment.
You'll probably want to refer to the lines in a specific version/commit. Otherwise the line numbers would change over time.
|
|
||
| <span style="text-decoration:underline;">Raw Kubernetes YAML:</span> | ||
|
|
||
| Argo CD also supports Kustomize and raw Kubernetes YAML. It can, however, apply raw YAML files directly, without using Kustomize. This can have side effects of Argo CD Applications not syncing right aways as resources need to be reapplied in a retry ([which can be set up per Application](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/application.yaml#L111-L116)), or use ordering built into Argo CD. Argo CD does the equivalent of `kubectl apply -f` or a `kustomzie build` if Kustomize is detected. This is by design as the Argo CD creators wanted to emulate the `kubectl` client as closely as possible. |
There was a problem hiding this comment.
| Argo CD also supports Kustomize and raw Kubernetes YAML. It can, however, apply raw YAML files directly, without using Kustomize. This can have side effects of Argo CD Applications not syncing right aways as resources need to be reapplied in a retry ([which can be set up per Application](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/application.yaml#L111-L116)), or use ordering built into Argo CD. Argo CD does the equivalent of `kubectl apply -f` or a `kustomzie build` if Kustomize is detected. This is by design as the Argo CD creators wanted to emulate the `kubectl` client as closely as possible. | |
| Argo CD also supports Kustomize and raw Kubernetes YAML. It can, however, apply raw YAML files directly, without using Kustomize. This can have side effects of Argo CD Applications not syncing right aways as resources need to be reapplied in a retry ([which can be set up per Application](https://github.com/argoproj/argo-cd/blob/master/docs/operator-manual/application.yaml#L111-L116)), or use ordering built into Argo CD. Argo CD does the equivalent of `kubectl apply -f` or a `kustomize build` if Kustomize is detected. This is by design as the Argo CD creators wanted to emulate the `kubectl` client as closely as possible. |
|
|
||
| ## Argo CD | ||
|
|
||
| Argo CD can also [integrate with a variety of Secret management tools](https://argo-cd.readthedocs.io/en/stable/operator-manual/secret-management/) to store these secrets in Git, safely. It leaves this choice up to the user. Some users have taken advantage of Argo CD’s plugin feature to create plugins, for example, the[ Argo CD Vault Plugin](https://github.com/IBM/argocd-vault-plugin). This “hands off” approach just treats secrets as another _Application_ in the cluster. |
There was a problem hiding this comment.
This is now in argoproj-labs. https://github.com/argoproj-labs/argocd-vault-plugin
|
|
||
| <span style="text-decoration:underline;">Raw Kubernetes YAML:</span> | ||
|
|
||
| Flux uses Kustomize heavily. Its[ Kustomize Controller](https://fluxcd.io/docs/components/kustomize/) renders and applies all of the manifests that Flux syncs from the source repository, even when using Raw Kubernetes YAML. Flux will create a `kustommization.yaml` file “on the fly”, if Kustomize wasn’t detected. |
There was a problem hiding this comment.
| Flux uses Kustomize heavily. Its[ Kustomize Controller](https://fluxcd.io/docs/components/kustomize/) renders and applies all of the manifests that Flux syncs from the source repository, even when using Raw Kubernetes YAML. Flux will create a `kustommization.yaml` file “on the fly”, if Kustomize wasn’t detected. | |
| Flux uses Kustomize heavily. Its[ Kustomize Controller](https://fluxcd.io/docs/components/kustomize/) renders and applies all of the manifests that Flux syncs from the source repository, even when using Raw Kubernetes YAML. Flux will create a `kustomization.yaml` file “on the fly”, if Kustomize wasn’t detected. |
|
Based on conversations and what was discussed in past meetings, I move to close this PR in favor of focusing more on validated patterns and best practices docs. Since this is my PR, I'm going to leave it open for a couple of days but will close it if I don't hear objections. |
Closing based on my previous message ^ Feel free to repoen if you disagree. |
This is the new stack blog that was started by Aubrey and myself (from Red Hat), that was meant to capitalize on the momentum of GitOpsCon NA 2021. One of the most asked question was "How does Argo CD and Flux Compare?", this was opened up for feedback and was collaborated on by various members of the GitOps Working Group.
After a few rounds of feedback and collaboration, we decided to open this up to the larger community. As the blog was getting quite large, we thought it best to move this as an official PR for a living in-depth comparison document by the GitOps Working Group. We hope to instead try to summarize the higher level technical differences in the blog. The goal being for the blog to point back to this PR as this will probably take some time to complete. This can also pave the way for additional comparisons between other tools in the GitOps ecosystem.
(If I missed any contributors, please add)
Contributors so far:
Additional Review Done By: