Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Argo CD unnecessary enforce sequential helm manifest generation for one chart #17518

Merged
merged 2 commits into from
Mar 13, 2024
Merged

fix: Argo CD unnecessary enforce sequential helm manifest generation for one chart #17518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
030552b
fix: Argo CD unnecessary enforce sequential helm manifest generation …
alexmt Mar 13, 2024
7ad38b2
Update docs/operator-manual/high_availability.md
todaywasawesome Mar 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/operator-manual/high_availability.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ Argo CD repo server maintains one repository clone locally and uses it for appli
Argo CD determines if manifest generation might change local files in the local repository clone based on the config management tool and application settings.
If the manifest generation has no side effects then requests are processed in parallel without a performance penalty. The following are known cases that might cause slowness and their workarounds:

* **Multiple Helm based applications pointing to the same directory in one Git repository:** ensure that your Helm chart doesn't have conditional
[dependencies](https://helm.sh/docs/chart_best_practices/dependencies/#conditions-and-tags) and create `.argocd-allow-concurrency` file in the chart directory.
* **Multiple Helm based applications pointing to the same directory in one Git repository:** for historical reasons Argo CD generates Helm manifests sequentially. To enable parallel generation set `ARGOCD_HELM_ALLOW_CONCURRENCY=true` to `argocd-repo-server` deployment or create `.argocd-allow-concurrency` file.
Future versions of Argo CD will enable this by default.

* **Multiple Custom plugin based applications:** avoid creating temporal files during manifest generation and create `.argocd-allow-concurrency` file in the app directory, or use the sidecar plugin option, which processes each application using a temporary copy of the repository.

Expand Down
26 changes: 14 additions & 12 deletions reposerver/repository/repository.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,36 +16,30 @@ import (
"strings"
"time"

"github.com/golang/protobuf/ptypes/empty"

kubeyaml "k8s.io/apimachinery/pkg/util/yaml"

"k8s.io/apimachinery/pkg/api/resource"

"github.com/argoproj/argo-cd/v2/common"
"github.com/argoproj/argo-cd/v2/util/io/files"
"github.com/argoproj/argo-cd/v2/util/manifeststream"

"github.com/Masterminds/semver/v3"
"github.com/TomOnTime/utfutil"
"github.com/argoproj/gitops-engine/pkg/utils/kube"
textutils "github.com/argoproj/gitops-engine/pkg/utils/text"
"github.com/argoproj/pkg/sync"
jsonpatch "github.com/evanphx/json-patch"
gogit "github.com/go-git/go-git/v5"
"github.com/golang/protobuf/ptypes/empty"
"github.com/google/go-jsonnet"
"github.com/google/uuid"
grpc_retry "github.com/grpc-ecosystem/go-grpc-middleware/retry"
log "github.com/sirupsen/logrus"
"golang.org/x/sync/semaphore"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
kubeyaml "k8s.io/apimachinery/pkg/util/yaml"
"sigs.k8s.io/yaml"

pluginclient "github.com/argoproj/argo-cd/v2/cmpserver/apiclient"
"github.com/argoproj/argo-cd/v2/common"
"github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"
"github.com/argoproj/argo-cd/v2/reposerver/apiclient"
"github.com/argoproj/argo-cd/v2/reposerver/cache"
Expand All @@ -54,14 +48,17 @@ import (
argopath "github.com/argoproj/argo-cd/v2/util/app/path"
"github.com/argoproj/argo-cd/v2/util/argo"
"github.com/argoproj/argo-cd/v2/util/cmp"
"github.com/argoproj/argo-cd/v2/util/env"
"github.com/argoproj/argo-cd/v2/util/git"
"github.com/argoproj/argo-cd/v2/util/glob"
"github.com/argoproj/argo-cd/v2/util/gpg"
"github.com/argoproj/argo-cd/v2/util/grpc"
"github.com/argoproj/argo-cd/v2/util/helm"
"github.com/argoproj/argo-cd/v2/util/io"
"github.com/argoproj/argo-cd/v2/util/io/files"
pathutil "github.com/argoproj/argo-cd/v2/util/io/path"
"github.com/argoproj/argo-cd/v2/util/kustomize"
"github.com/argoproj/argo-cd/v2/util/manifeststream"
"github.com/argoproj/argo-cd/v2/util/text"
)

Expand All @@ -74,7 +71,12 @@ const (
ociPrefix = "oci://"
)

var ErrExceededMaxCombinedManifestFileSize = errors.New("exceeded max combined manifest file size")
var (
ErrExceededMaxCombinedManifestFileSize = errors.New("exceeded max combined manifest file size")
// helmConcurrencyDefault if true then helm concurrent manifest generation is enabled
// TODO: remove env variable and usage of .argocd-allow-concurrency once we are sure that it is safe to enable it by default
helmConcurrencyDefault = env.ParseBoolFromEnv("ARGOCD_HELM_ALLOW_CONCURRENCY", false)
)

// Service implements ManifestService interface
type Service struct {
Expand Down Expand Up @@ -1096,7 +1098,7 @@ func isSourcePermitted(url string, repos []string) bool {
}

func helmTemplate(appPath string, repoRoot string, env *v1alpha1.Env, q *apiclient.ManifestRequest, isLocal bool, gitRepoPaths io.TempPaths) ([]*unstructured.Unstructured, error) {
concurrencyAllowed := isConcurrencyAllowed(appPath)
concurrencyAllowed := helmConcurrencyDefault || isConcurrencyAllowed(appPath)
if !concurrencyAllowed {
manifestGenerateLock.Lock(appPath)
defer manifestGenerateLock.Unlock(appPath)
Expand Down
Loading