Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(opensearch): domain add more selector and references #1337

Merged
merged 1 commit into from
Jun 4, 2024
Merged

feat(opensearch): domain add more selector and references #1337

merged 1 commit into from
Jun 4, 2024

Conversation

haarchri
Copy link
Member

@haarchri haarchri commented Jun 1, 2024

Description of your changes

Fixes #

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable to ensure this PR is ready for review.
  • Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

we want to migrate from community provider - this PR adds missing Selector & References:

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-encryptionAtRestOptions-kmsKeyIdRef

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-encryptionAtRestOptions-kmsKeyIdSelector

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-securityGroupIdRefs

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-securityGroupIdSelector

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-subnetIdRefs

https://marketplace.upbound.io/providers/crossplane-contrib/provider-aws/v0.48.1/resources/opensearchservice.aws.crossplane.io/Domain/v1alpha1#doc:spec-forProvider-vpcOptions-subnetIdSelector

@ytsarev
Copy link
Collaborator

ytsarev commented Jun 2, 2024

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri
Copy link
Member Author

haarchri commented Jun 2, 2024

@ytsarev

 [{0 OpenSearch
    logger.go:42: 08:44:38 | case/0-apply |         Domain "example-domain-name" already exists  []}]'

ulucinar
ulucinar approved these changes Jun 4, 2024
View reviewed changes
Copy link
Collaborator

@ulucinar ulucinar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @haarchri, lgtm.

@ytsarev
Copy link
Collaborator

ytsarev commented Jun 4, 2024

@haarchri can you please add ${Rand.RFC1123Subdomain} to the failing example as in https://github.com/crossplane/uptest?tab=readme-ov-file#injecting-dynamic-values-and-datasource so we would avoid clashes during the e2e test of the Domain?

@ytsarev
Copy link
Collaborator

ytsarev commented Jun 4, 2024

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri
Copy link
Member Author

haarchri commented Jun 4, 2024

2024-06-04T11:13:01.8825601Z logger.go:42: 11:13:01 | case/0-apply | - lastTransitionTime: "2024-06-04T11:08:55Z" 2024-06-04T11:13:01.8827013Z logger.go:42: 11:13:01 | case/0-apply | message: 'async create failed: failed to create the resource: [{0 creating OpenSearch 2024-06-04T11:13:01.8828741Z logger.go:42: 11:13:01 | case/0-apply | Domain: ValidationException: 1 validation error detected: Value ''example-domain-name-op-3j4sno06'' 2024-06-04T11:13:01.8830434Z logger.go:42: 11:13:01 | case/0-apply | at ''domainName'' failed to satisfy constraint: Member must have length less 2024-06-04T11:13:01.8831647Z logger.go:42: 11:13:01 | case/0-apply | than or equal to 28 []}]'

@haarchri
feat(opensearch): add ref/selector for opensearch domains
4ee826c 
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
@haarchri haarchri force-pushed the feature/opensearch-domain-ref-selectors branch from 6d828cd to 4ee826c Compare June 4, 2024 12:53
@ytsarev
Copy link
Collaborator

ytsarev commented Jun 4, 2024

/test-examples

@haarchri
Copy link
Member Author

haarchri commented Jun 4, 2024

@ytsarev can we test the following example ? examples/opensearch/v1beta1/domain.yaml

@ytsarev
Copy link
Collaborator

ytsarev commented Jun 4, 2024

/test-examples="examples/opensearch/v1beta1/domain.yaml"

@haarchri
Copy link
Member Author

haarchri commented Jun 4, 2024

mhm - any ideas ?

    logger.go:42: 14:40:31 | case/0-apply |     - lastTransitionTime: "2024-06-04T14:40:11Z"
    logger.go:42: 14:40:31 | case/0-apply |       message: 'update failed: async update failed: failed to update the resource:
    logger.go:42: 14:40:31 | case/0-apply |         [{0 updating OpenSearch Domain (arn:aws:es:us-west-1:153891904029:domain/op-jz2ysr4s):
    logger.go:42: 14:40:31 | case/0-apply |         ValidationException: A change/update is in progress. Please wait for it to
    logger.go:42: 14:40:31 | case/0-apply |         complete before requesting another change.  []}]'

@haarchri
Copy link
Member Author

haarchri commented Jun 4, 2024

think its not realted to my change:

 kubectl describe domain.opensearch.aws.upbound.io/example
Name:         example
Namespace:    
Labels:       testing.upbound.io/example-name=example
Annotations:  crossplane.io/external-create-pending: 2024-06-04T15:36:03Z
              crossplane.io/external-create-succeeded: 2024-06-04T15:36:03Z
              crossplane.io/external-name: arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
              meta.upbound.io/example-id: opensearch/v1beta1/domain
              upjet.upbound.io/test: true
API Version:  opensearch.aws.upbound.io/v1beta1
Kind:         Domain
Metadata:
  Creation Timestamp:  2024-06-04T15:36:00Z
  Finalizers:
    finalizer.managedresource.crossplane.io
  Generation:        3
  Resource Version:  3355
  UID:               e2af1582-b526-40be-8064-0cec18096bf9
Spec:
  Deletion Policy:  Delete
  For Provider:
    Auto Tune Options:
      Desired State:        ENABLED
      Rollback On Disable:  NO_ROLLBACK
    Cluster Config:
      Instance Type:  m4.large.search
    Cognito Options:
      Identity Pool Id:  
      Role Arn:          
      User Pool Id:      
    Domain Endpoint Options:
      Tls Security Policy:  Policy-Min-TLS-1-2-2019-07
    Domain Name:            op-jqg7nlkn
    Ebs Options:
      Ebs Enabled:  true
      Volume Size:  10
      Volume Type:  gp2
    Encrypt At Rest:
      Enabled:       false
    Engine Version:  OpenSearch_1.0
    Node To Node Encryption:
      Enabled:  false
    Off Peak Window Options:
      Enabled:  true
      Off Peak Window:
        Window Start Time:
          Hours:  5
    Region:       us-west-1
    Snapshot Options:
      Automated Snapshot Start Hour:  0
    Software Update Options:
    Tags:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
  Init Provider:
  Management Policies:
    *
  Provider Config Ref:
    Name:  default
Status:
  At Provider:
    Arn:  arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
    Auto Tune Options:
      Desired State:        ENABLED
      Rollback On Disable:  NO_ROLLBACK
      Use Off Peak Window:  false
    Cluster Config:
      Cold Storage Options:
        Enabled:                      false
      Dedicated Master Count:         0
      Dedicated Master Enabled:       false
      Dedicated Master Type:          
      Instance Count:                 1
      Instance Type:                  m4.large.search
      Multi Az With Standby Enabled:  false
      Warm Count:                     0
      Warm Enabled:                   false
      Warm Type:                      
      Zone Awareness Enabled:         false
    Cognito Options:
      Enabled:           false
      Identity Pool Id:  
      Role Arn:          
      User Pool Id:      
    Dashboard Endpoint:  search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com/_dashboards
    Domain Endpoint Options:
      Custom Endpoint:                  
      Custom Endpoint Certificate Arn:  
      Custom Endpoint Enabled:          false
      Enforce Https:                    true
      Tls Security Policy:              Policy-Min-TLS-1-2-2019-07
    Domain Id:                          609897127049/op-jqg7nlkn
    Domain Name:                        op-jqg7nlkn
    Ebs Options:
      Ebs Enabled:  true
      Iops:         0
      Throughput:   0
      Volume Size:  10
      Volume Type:  gp2
    Encrypt At Rest:
      Enabled:        false
      Kms Key Id:     
    Endpoint:         search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com
    Engine Version:   OpenSearch_1.0
    Id:               arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn
    Kibana Endpoint:  search-op-jqg7nlkn-phjcqlnsrmukuvxrg4yaixldn4.us-west-1.es.amazonaws.com/_plugin/kibana/
    Node To Node Encryption:
      Enabled:  false
    Off Peak Window Options:
      Enabled:  true
      Off Peak Window:
        Window Start Time:
          Hours:    5
          Minutes:  0
    Snapshot Options:
      Automated Snapshot Start Hour:  0
    Software Update Options:
      Auto Software Update Enabled:  false
    Tags:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
    Tags All:
      Crossplane - Kind:            domain.opensearch.aws.upbound.io
      Crossplane - Name:            example
      Crossplane - Providerconfig:  default
  Conditions:
    Last Transition Time:  2024-06-04T16:02:36Z
    Reason:                Available
    Status:                True
    Type:                  Ready
    Last Transition Time:  2024-06-04T16:15:47Z
    Reason:                ReconcileSuccess
    Status:                True
    Type:                  Synced
    Last Transition Time:  2024-06-04T15:47:33Z
    Reason:                Success
    Status:                True
    Type:                  LastAsyncOperation
Events:
  Type    Reason                   Age                   From                                                    Message
  ----    ------                   ----                  ----                                                    -------
  Normal  CreatedExternalResource  42m                   managed/opensearch.aws.upbound.io/v1beta1, kind=domain  Successfully requested creation of external resource
  Normal  UpdatedExternalResource  2m47s (x73 over 30m)  managed/opensearch.aws.upbound.io/v1beta1, kind=domain  Successfully requested update of external resource

2024-06-04T16:19:16Z	DEBUG	provider-aws	Diff detected	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"advanced_security_options.#\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3355", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "requeue-after": "2024-06-04T16:29:14Z"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Async update starting...	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:16Z	DEBUG	provider-aws	Updating the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Async update ended.	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "error": null, "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Reconcile request has been requeued.	{"gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "name": "example", "rateLimiterName": "", "when": "0s"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Reconciling	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Connecting to the service provider	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:20Z	DEBUG	provider-aws	Observing the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Diff detected	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"advanced_security_options.#\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Successfully requested update of external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3355", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "requeue-after": "2024-06-04T16:29:45Z"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Async update starting...	{"trackerUID": "e2af1582-b526-40be-8064-0cec18096bf9", "resourceName": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "tfID": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn"}
2024-06-04T16:19:21Z	DEBUG	provider-aws	Updating the external resource	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Calling the inner handler for Update event.	{"gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "name": "example", "queueLength": 0}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Reconciling	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Connecting to the service provider	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	ongoing async operation	{"uid": "e2af1582-b526-40be-8064-0cec18096bf9", "name": "example", "gvk": "opensearch.aws.upbound.io/v1beta1, Kind=Domain", "opType": "update"}
2024-06-04T16:19:22Z	DEBUG	provider-aws	Cannot delete external resource	{"controller": "managed/opensearch.aws.upbound.io/v1beta1, kind=domain", "request": {"name":"example"}, "uid": "e2af1582-b526-40be-8064-0cec18096bf9", "version": "3846", "external-name": "arn:aws:es:us-west-1:609897127049:domain/op-jqg7nlkn", "deletion-timestamp": "2024-06-04 16:19:22 +0000 UTC", "error": "update operation that started at 2024-06-04 16:19:21.830019794 +0000 UTC m=+1214.870558596 is still running", "errorVerbose": "update operation that started at 2024-06-04 16:19:21.830019794 +0000 UTC m=+1214.870558596 is still running\ngh.neting.cc/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Delete\n\tgh.neting.cc/crossplane/upjet@v1.4.0/pkg/controller/external_async_tfpluginsdk.go:193\ngh.neting.cc/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgh.neting.cc/crossplane/crossplane-runtime@v1.16.0-rc.2.0.20240510094504-3f697876fa57/pkg/reconciler/managed/reconciler.go:954\ngh.neting.cc/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgh.neting.cc/crossplane/crossplane-runtime@v1.16.0-rc.2.0.20240510094504-3f697876fa57/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1222"}

@ytsarev
Copy link
Collaborator

ytsarev commented Jun 4, 2024

i trust we can merge the selectors and address the MR behaviour separately

@ytsarev ytsarev merged commit 4b4cac4 into crossplane-contrib:main Jun 4, 2024
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ulucinar ulucinar ulucinar approved these changes

@ytsarev ytsarev ytsarev approved these changes

@sergenyalcin sergenyalcin Awaiting requested review from sergenyalcin sergenyalcin is a code owner

@turkenf turkenf Awaiting requested review from turkenf turkenf is a code owner

@mbbush mbbush Awaiting requested review from mbbush mbbush is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@haarchri @ytsarev @ulucinar