kustomization resources are silently dropped if there is an typo an yaml is no valid kubernetes yaml anymore

[haarchri]

kustomization resources are silently dropped if there is an typo an yaml is no valid kubernetes yaml anymore

let's say we have a repo with lot of yaml files and kustomization.yaml inside - and the commit id ismaster/db9bb76889ce22441f2e63e2d79702b97f4e35f1

---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: crossplane-provider-aws
spec:
  package: crossplane/provider-aws:v0.23.0
  controllerConfigRef:
    name: provider-aws

we see the following in the logs - everything okay and expected:

{"level":"info","ts":"2022-01-20T11:03:48.030Z","logger":"controller.kustomization","msg":"server-side apply completed","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system","output":{"Cluster/default/central-crossplane-prod":"unchanged","Configuration/crossplane-sop":"unchanged","ControllerConfig/provider-aws":"unchanged","Kustomization/flux-system/sources":"unchanged","Network/default/central-crossplane-prod-network":"unchanged","Provider/crossplane-provider-aws":"unchanged","ProviderConfig/aws-provider-1234567891012":"unchanged"}}

with the next commit the yaml file content changed because of a typo to apiVersion <-> piVersion - and the commit id is now master/f218296faac11060a98b6b3a9babfcaabf934553

---
piVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: crossplane-provider-aws
spec:
  package: crossplane/provider-aws:v0.23.0
  controllerConfigRef:
    name: provider-aws

we see the following in the logs:

{"level":"info","ts":"2022-01-20T12:03:48.030Z","logger":"controller.kustomization","msg":"server-side apply completed","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system","output":{"Cluster/default/central-crossplane-prod":"unchanged","Configuration/crossplane-sop":"unchanged","ControllerConfig/provider-aws":"unchanged","Kustomization/flux-system/sources":"unchanged","Network/default/central-crossplane-prod-network":"unchanged","Provider/crossplane-provider-aws":"unchanged","ProviderConfig/aws-provider-1234567891012":"unchanged"}}
{"level":"info","ts":"2022-01-20T12:03:48.030Z","logger":"controller.kustomization","msg":"server-side apply completed","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system","output":{"Cluster/default/central-crossplane-prod":"unchanged","Configuration/crossplane-sop":"unchanged","ControllerConfig/provider-aws":"unchanged","Kustomization/flux-system/sources":"unchanged","Network/default/central-crossplane-prod-network":"unchanged","Provider/crossplane-provider-aws":"unchanged","ProviderConfig/aws-provider-1234567891012":"unchanged"}}
{"level":"info","ts":"2022-01-20T12:03:48.041Z","logger":"controller.kustomization","msg":"Reconciliation finished in 1.985765791s, next run in 10m0s","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system","revision":"master/f218296faac11060a98b6b3a9babfcaabf934553"}

in next reconcile loop we see that the resource is silently dropped - without an issue in kustomization build

{"level":"info","ts":"2022-01-20T12:13:46.331Z","logger":"controller.kustomization","msg":"server-side apply completed","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system","output":{"Cluster/default/central-crossplane-prod":"unchanged","Configuration/crossplane-sop":"unchanged","ControllerConfig/provider-aws":"unchanged","Kustomization/flux-system/sources":"unchanged","Network/default/central-crossplane-prod-network":"unchanged","ProviderConfig/aws-provider-1234567891012":"unchanged"}}
{"level":"info","ts":"2022-01-20T12:13:46.346Z","logger":"controller.kustomization","msg":"garbage collection completed: Provider/crossplane-provider-aws deleted","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"central-crossplane-prod","namespace":"flux-system"}

if we run the kustomization build locally we can see the following issue:

kustomize build --load-restrictor=LoadRestrictionsNone --reorder=legacy . \
| kubectl apply --server-side --dry-run=server -f-

error validating "STDIN": error validating data: apiVersion not set; if you choose to ignore these errors, turn validation off with --validate=false
Apply failed with 1 conflict: conflict with "kustomize-controller": .spec.ports
Please review the fields above--they currently have other managers. Here
are the ways you can resolve this warning:
* If you intend to manage all of these fields, please re-run the apply
  command with the `--force-conflicts` flag.
* If you do not intend to manage all of the fields, please edit your
  manifest to remove references to the fields that should keep their
  current managers.
* You may co-own fields by updating your manifest to match the existing
  value; in this case, you'll become the manager if the other manager(s)
  stop managing the field (remove it from their configuration).
See https://kubernetes.io/docs/reference/using-api/server-side-apply/#conflicts

our kustomization looks like:

Name:         central-crossplane-prod
Namespace:    flux-system
Labels:       app.kubernetes.io/instance=flux-system
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/part-of=flux
              helm.sh/chart=flux2-sync-0.3.6
Annotations:  meta.helm.sh/release-name: central-crossplane-prod
              meta.helm.sh/release-namespace: flux-system
              reconcile.fluxcd.io/requestedAt: 2022-01-14T15:16:19.55198+01:00
API Version:  kustomize.toolkit.fluxcd.io/v1beta2
Kind:         Kustomization
Spec:
  Decryption:
    Provider:  sops
  Force:       false
  Interval:    10m0s
  Path:        ./base
  Prune:       true
  Source Ref:
    Kind:  GitRepository
    Name:  central-crossplane-prod
Status:
  Conditions:
    Last Transition Time:  2022-01-20T14:34:08Z
    Message:             Applied revision: master/f218296faac11060a98b6b3a9babfcaabf934553
    Reason:                ReconciliationSucceeded
    Status:                True
    Type:                  Ready