Experimental

This is an experimental release which containing the most recent commits from the main branch as of commit: b06760b.

This release might not be stable. Use at your own risk.

⚠️ The provided YAML manifest does not configure any CRDs to install by default, but is required.
You must specify the CRDs that you want to use as part of crdPattern, for example 'resources.azure.com/*;containerservice.azure.com/*;keyvault.azure.com/*;managedidentity.azure.com/*;apimanagement.azure.com/*'.

The recommended way to supply crdPattern is using asoctl template:

asoctl export template --source https://github.com/Azure/azure-service-operator/releases/download/experimental/azureserviceoperator_experimental.yaml --crd-pattern "<pattern>" | kubectl apply -f -

This release is only intended for developers wishing to try out the latest features, some of which may not be fully implemented.

It is not recommended to run the experimental release for a long period of time, as the docker image referenced by the deployment is
mcr.microsoft.com/k8s/azureserviceoperator:experimental, which is always being updated. Test what you want to test and then uninstall the operator.
Running the operator for long periods of time on the experimental tag is not supported and will likely cause problems eventually.

v2.10.0

Release notes

Breaking changes

MachineLearningServices/WorkspacesCompute properties marked as a SecretReference

The below properties on WorkspacesCompute has been changed from a string to a SecretReference
We always try to avoid breaking changes, but in this case, allowing raw passwords in the spec is a security problem and as such we've
decided to make a break to correct this issue.

Affected Properties:

• SslConfiguration.Key
• SslConfiguration.Cert
• VirtualMachineSshCredentials.PrivateKeyData
• VirtualMachineSshCredentials.PublicKeyData
• DatabricksProperties.DatabricksAccessToken

For more details see the breaking changes document.

Upcoming Breaking changes

We've made a number of breaking changes to properties to ensure they correctly use Secrets to store secret values.
To reduce the likelihood of future breaking changes of this type, we are working to improve our automated detection of properties that should be treated as secrets, see PR #4289.

New resources

• Add support for new machinelearningservices API version v1api20240401 (#4237)
• Add support for new machinelearningservices Registry resource (#4339)
• Add support for new kubernetesconfiguration FluxConfiguration resource (#4275)
• Add support for new cache API version v1api20230801 (#4287)
• Add support for new network DnsForwardingRuleSetsVirtualNetworkLink resource (#4345)
• Add support for new network ApplicationSecurityGroup group resource (#4342)
• Add support for new network ApplicationGatewayWebApplicationFirewallPolicies resource (#4238)

Features

• Allow the data encryption keys for PostgreSQL Flexible server to be configured via ConfigMap (#4311)

Improvements

• Allow significantly more Helm chart configuration (#4211, #4207)
• Remove pre-upgrade check (#4217)
• Updated numerous dependencies
• Improve controller pod security context restrictions (#4242)
• Release lease when the pod is terminated for faster pod shutdown (#4250)
• Improve the spec and status names for some resources (breaking for Go pkg consumers) (#4341)
• asoctl: Add some validation of crd-pattern to export template command (#4245)
• asoctl: Allow reuse of import command via Go code (#4243)
• asoctl: Allow number of simultaneous workers to be configured (#4257)
• asoctl: Reduce noise when importing PostgreSQL Flexible Server Configurations (#4279)
• asoctl: Reduce noise when importing MySQL Flexible Server Configurations (#4282)
• asoctl: Built-in role definitions should be skiped by import command (#4263)

Bug fixes

• Fix bug where a log could cause a panic (#4249)
• Fix bug where ASO's NetworkPolicy didn't allow egress to SqlServer's default port (#4283)
• Fix bug where ScheduledQueryRules could get stuck reconciling when encountering a BadRequest (#4312)
• Fix bug where storage account bypass parameter did not accept all allowed values (#4328)
• Fix bug where some ConfigMap and Secret references could fail to be exported (#4330)
• asoctl: Fix bug where resource names could be generated which where not valid in Kubernetes (#4244)
• asoctl: Fix bug where FrontDoor resources couldn't be imported (#4262)
• asoctl: Fix deadlock and ux bugs (#4270)

Documentation

• Add missing SQL User documentation (#4230)
• Add docs for non-podidentity managed identity (#4227)
• Add some additional documentation about the controller metrics endpoint (#4273)
• Update redis migration docs with correct redis key (#4297)
• Add ArgoCD FAQ item (#4309)

Full Changelog: v2.9.0...v2.10.0

v2.9.0

Release notes

Breaking changes

v20230202preview ManagedCluster was removed

This removal was due to the underlying Azure API version being deprecated.

Other versions will continue to be supported. We recommend you move to use a different CRD version to avoid
seeing errors from Azure due to the 2023-02-02-preview API being deprecated. This change was performed in PR #4182.

See the breaking change documentation for 2.9.0 for more details on how to deal with this, especially if you are using or have used TrustedAccessRoleBinding.

The RoleAssignment spec.delegatedManagedIdentityResourceId field has changed

This field was a string containing an ARM ID previously. It is now a genruntime.ResourceReference.

This field is not commonly used. If your RoleAssignment resources don't set this field, there is no action needed.

If your cluster has RoleAssignment resources that set this field:

• Set the serviceoperator.azure.com/reconcile-policy: detach-on-delete annotation.
• Issue kubectl delete for those RoleAssignments. They will be deleted in ASO but not Azure.
• Upgrade ASO.
• Re-apply the RoleAssignment resource with the new delegatedManagedIdentityResourceReference field set.

Upcoming Breaking changes

None.

New resources

• Add support for new Disk API version v20240302 (#4171)
• Add support for Azure RedhatOpenshift Clusters (#4152)

Features

• Remove default operator-scoped rate limiter and add various scaling configuration options (#4158)
• Add ability to configure RoleAssignments UUID generation algorithm. Allowed options are random and stable (#4196)
• Allow custom OperatorSpec properties to be defined in generator configuration (#4175)

Improvements

• Updated numerous dependencies
• Fix pre-release-test (#4128)
• Update local tests and CI tests to use az login token or managed identity, not service principal (#4003)

Bug fixes

• Fix bug where KeyVault with ARMID owner couldn't be recovered (#4127)
• Fix bug where 'none' crd-management mode blocked pod start (#4151)
• Fix bug where BackupVaults/BackupInstance would report ready condition before it was actually ready (#4180)
• Fix bug where owner.armId could be changed (#4193)

Documentation

• Add missing aso-controller-settings documentation (#4160)
• Add contact details to our documentation site (#4190)

Full Changelog: v2.8.0...v2.9.0

v2.8.0

Release notes

Breaking changes

Use "never" rather than "" to prevent syncing for AZURE_SYNC_PERIOD

The documentation always said that an AZURE_SYNC_PERIOD of "" meant to use the default value (15m), but
in actuality in the code "" meant never sync. This corrects the behavior to be as documented. After this release,
the behavior is as follows:

AZURE_SYNC_PERIOD	Meaning
omitted/not set	Use default value (15m)
""	Use default value (15m)
"never"	Do not sync

If you intend to prevent ASO from periodically syncing with Azure, set AZURE_SYNC_PERIOD to "never".
For more information, see #3965.

containerservice.azure.com ManagedCluster property .spec.properties.windowsProfile.adminPassword is now a secret reference rather than a string

The spec.properties.windowsProfile.adminPassword on ManagedCluster has been changed from a string to a
SecretReference.
We try to avoid breaking changes, but in this case, allowing raw passwords in the spec is a security
problem and as such we've decided to make a break to correct this issue.
Action required: If the containerservice.azure.com/ManagedCluster resource is used in your cluster
and the spec.properties.windowsProfile.adminPassword property is set, follow the steps in the
breaking changes document

Upcoming Breaking changes

v20230202preview ManagedCluster will be removed, due to underlying Azure API version deprecation

Other versions will continue to be supported. We recommend you move to use a different CRD version to avoid
seeing errors from Azure due to the 2023-02-02-preview API being deprecated.

New resources

• Support new API version of DocumentDB resources (#3967)
• Support Prom Rule groups and Azure Monitor accounts (workspaces) (#4052)
• Support for RoleDefinition resource (#4067)
• Support new AKS preview API version 2024-04-02-preview (#4086)

Features

• Support multiple clouds in asoctl (#4033)
• Support to specify image pull secrets in ASOv2 helm chart (#4116)
• Expose default primary PrivateEndpoint PrivateIpAddress (#4107)

Improvements

• Updated numerous Golang dependencies
• Update local tests and CI tests to use az login token or managed identity, not service principal (#4003)

Bug fixes

• Fix bug where AZURE_USER_AGENT_SUFFIX was not getting set (#4011)
• Fix bug where AZURE_SYNC_PERIOD "" meant "never sync" when it should have meant "use the default sync period" (see also the breaking changes) (#4049)
• Fix bug where ManagedCluster WindowsProfile password was a string (see also the breaking changes) (#4091)
• Fix bug where asoctl wasn't case-correcting enums when it should have been (#4109)
• Fix bug where asoctl would sometimes copy the wrong value between two properties with the same name if they were flattened, such as .type and .properties.type (#4108)

Documentation

• Improve secret rotation documentation, based on user pattern (#4110)

Full Changelog: v2.7.0...v2.8.0

v2.7.0

Release notes

Breaking changes

Metrics endpoint has changed from 8080 to 8443 and now requires HTTPS

ASO no longer depends on kube-rbac-proxy and now correctly secures the metrics endpoint by default.

These options are configured via the following options in the Helm chart:

    --set metrics.secure=true/false (default: true)
    --set metrics.address=0.0.0.0:8443 (default)

For more details about how to scrape metrics, see the metrics guide.

Secret or ConfigMap values which cannot be written now trigger a reconcile error

Previously, if ASO couldn't find the corresponding secret/configmap value, it would just skip creating it with no error. This is almost
never what users expect: if they ask for a specific ConfigMap or Secret value to be exported they expect it to actually get exported. ASO now
behaves as expected in cases where it cannot find the value to export and will set an appropriate Ready condition warning.

See #3925 for more details.

Upcoming Breaking changes

None

New resources

• Support new MySQL API version 2023-06-30 (#3905)
• Support new Insight/Webtest API version 2022-06-15 (#3911)
• New resource Backup Instance for Microsoft.DataProtection (#3736)

Features

• Expose pprof endpoint at metrics URL when setting --set metrics.profiling=true/false (default: false) (#3833)
• Managedidentity operatorspec supports export to a secret (#3937)
• Add asoctl template command (#3968)

Improvements

• Updated numerous Golang dependencies
• Support exporting eventhub keys (#3882)
• Add namespace, label, and annotation support to asoctl import (#3884)

Bug fixes

• Fix bug where Reconciling condition would sometimes fail to overwrite AzureResourceNotFound (#3834)
• Fix bug where documentdb capacity failures had a very hard to understand error (#3906)
• Fix bug where user requested secrets or configmaps were not emtited in some edge cases (#3925)
• Fix asoctl bug that could prevent imported resources from being applied because of letter case differences in responses from ARM (#3880)
• Fix asoctl bug where attempts to list the extension resource kubernetestconfiguration/extension could abort the import (#3853)
• Fix asoctl panic when resource types case mismatched (#3862)

Documentation

• Add ASOv1 to ASOv2 migration guide (#3898)
• Improve various small doc nits (#3909)
• Add best practices documentation and update other docs (#3938)
• Improve ownership, adoption, and FAQ documentation (#3966)

External Contributors

• @mayankagg9722 made their first contribution in #3736
• @mehighlow

Full Changelog: v2.6.0...v2.7.0

v2.6.0

Release notes

Breaking changes

VirtualMachineScaleSet ProtectedSettings changed to SecretReference (#3026)

We always try to avoid breaking changes, but in this case, allowing raw passwords in the spec is a security problem and as such we've
decided to take a break to correct this issue.

Action required: If the Compute/VirtualMachineScaleSet resource is installed in your cluster and the VirtualMachineProfile.ExtensionProfile.Extension.ProtectedSettings property is set on your VirtualMachineScaleSet resource, follow the steps in breaking changes document.

Upcoming Breaking changes

None

New resources

• Microsoft.PostgreSQL 2023-06-01-preview API version for FlexibleServer, FlexibleServersConfiguration, FlexibleServersDatabase and FlexibleServersFirewallRule (#3686)
• Microsoft.ApiManagement 2023-05-01-preview version for Service, Backend, NamedValue, Subscription, Api, Product, VersionSet, Policy, and PolicyFragment (#3673)
• Microsoft.ApiManagement ProductAPI, ProductPolicy, ApiVersionSet, AuthorizationProvider, AuthorizationProvidersAuthorization and AuthorizationProvidersAuthorizationsAccessPolicy (#3552) (#3644)
• Microsoft.ContainerService 2023-11-02-preview API version for ManagedClusters
• Microsoft.Compute VirtualMachine/Extension and VirtualMachineScaleSet/Extension
• Microsoft.CDN profiles/customDomains, profiles/afdEndpoints, profiles/originGroups, profiles/originGroups/origins, profiles/afdEndpoints/routes, profiles/ruleSets, profiles/ruleSets/rule, profiles/secrets and profiles/securityPolicies (#3785)
• Microsoft.Network FrontDoorWebApplicationFirewallPolicies (#3785)
• Azure SQL User (#3701)

Features

• Provide experimental ASO image built from main (#3699)
• Add ConfigurationProtectedSettings to KubernetesConfiguration/Extension resource (#3752)
• Add affinity and tolerations to ASOv2 helm chart (#3765)
• Export EventGrid/Topic Endpoint to ConfigMap (#3766)
• Export SignalR keys to Secret (#3698)
• Arm64 support for developer setup (#3687)

Improvements

• Improve for reconcile exit logging (#3755)
• Avoid superfluous error log in DELETE case (#3751)

Documentation

• Improve our documentation for Dev Setup (#3041)
• Include hand-crafted resources in documentation indexes (#3055)
• Update Managed Identity documentation (#3071)
• Add CRD pattern docs for each group (#3147)

External Contributors

• @ansulgoenka
• @mehighlow
• @tomkerkhove
• @ross-p-smith

Full Changelog: v2.5.0...v2.6.0

v2.5.0

Release Notes

Upcoming Breaking Changes

None

New Resources

• Update Microsoft.Keyvault to support the newer 2023-07-01 API version (#3605)
• Update Microsoft.Storage to support the newer 2023-01-01 API version (#3613 )
• Update Microsoft.ManagedIdentity to support the newer 2023-01-31 API version (#3606)
• Update Microsoft.PostgreSQL to support the newer 2022-12-01 API version (#3593)
• Update Microsoft.ContainerService to support the newer 2023-10-01 API version (#3629)

Features

• Add owner-name and owner-group-kind label on resources (#3608)
• Add extension for Microsoft.Eventgrid/Topic to export keys (#3633)
• Relax enum requirement for Microsoft.Compute/VirtualMachine (#3609)
• Add validation to catch use of armID as name (#3621)

Bug Fixes

• Controller does not allow to create child objects in different subscription anymore (#3546)
• Controller now uses HEAD if GET is not available (#3530)

Full Changelog: v2.4.0...v2.5.0

v2.4.0

Release Notes

Breaking Changes

Beta CRD deprecation

Beta versions of the CRDs have been removed this release.
You cannot successfully upgrade to v2.4.0 until you have followed our migration guide.
Fresh installations of v2.4.0 are unaffected.

Upcoming Breaking Changes

None

New Resources

• Add support for containerservice Fleet (#3336)
• Add support for networking TrafficManagerProfile and TrafficManagerProfileEndpoints (#3326)
• Add support for apimanagement Service, Backend, NamedValue, Subscription, Api, Product, VersionSet, Policy, and PolicyFragment (#3368)
• Add support for insights ActionGroups, MetricAlert, AutoscaleSetting, and ScheduledQueryRule (#3340)
• Add support for networking ApplicationGateway (#3176)
• Add support for kubernetesconfiguration Extensions (#3528)
• Update authorization to support the newer v2022-04-01 API version (#3504)

Tools

• Update asoctl clean crds command to clean beta CRDs (#3418)

Features

• Publish multi architecture docker images using buildx (#3355)
• Add createOrRecover and purgeThenCreate modes to KeyVaults, to encourage IAC patterns (#3357)
• Add a cmdline flag to control CRD management (#3445)
• Allow to customize webhook server port and cert dir (#3442)

Bug Fixes

• Fix containerservice ManagedCluster could get stuck due to transient CustomKubeletIdentityMissingPermissionError (#3286)
• Fix containerservice ManagedCluster and AgentPool do not correctly clear collections that had previously been set (#3407, #3540)
• Fix incorrect string case on namespace variable in Helm Chart (#3440)
• Fix resource doesn't re-reconcile if edited again during an update (#3468)

Documentation

• Add link to redis sample (#3317)
• Fix URL to github project on website (#3324)
• Improve FAQ and other documentation (#3302)
• Clarify MySQL AAD requirements (#3349)
• Document createOrRecover and purgeThenCreate KeyVault modes (#3400)
• Use createOrRecover for keyvault samples (#3370)
• Update documentation around CRD removal (#3422)

New Contributors

• @serbrech made their first contribution in #3324
• @nibooy made their first contribution in #3336
• @ross-p-smith made their first contribution in #3368
• @vincepri made their first contribution in #3442
• @mathewpeterson made their first contribution in #3440
• @vimorra made their first contribution in #3176
• @adamdougal made their first contribution in #3470

Full Changelog: v2.3.0...v2.4.0

v2.3.0

Release Notes

Breaking Changes

None.

Upcoming Breaking Changes

Beta CRD deprecation

Beta CRD versions (any version with v1beta prefix) will be deprecated no sooner than v2.4.0. We recommend you start using v1api prefixed versions now. You can easily swap from a v1beta version to a v1api version by just replacing v1beta with v1api in your CRD YAML.

New Resources

• Add support for compute/DiskEncryptionSets (#3211, resolves #2927)
• Support new versions of cache RP by (#3206)
• Update versions of ServiceBus (#3216, resolves #3143)

Tools

• asoctl: Redact empty status properties from when importing resources (#3180, resolves #3163)
• generator: Improve logging output (#2964, resolves #2853)

Features

• Refactor constants from internal/.. to pkg/common package to allow reuse of ASO as a library (#3171, resolves #3149)
• Add support for NetworkPolicies in v2 Helm chart by @tongpu (#3164, resolves #3160)
• Add new labels app.kubernetes.io/name and app.kubernetes.io/version (#3184)
• Add pod identity support for namespaces and per-resource scoped auth (#3187, resolves #3215)
• Make helm chart version SemVer 2 compatible (#3222, resolves #3189)
• Support arbitrary ARM ID owners (#3245)

Bug Fixes

• Fix Workload ID docs by @nojnhuh (#3156)
• Classify SQL "PasswordNotComplex" as fatal (#3185)
• Fix for asoctl omitting some child containers when importing Storage Accounts by using fully qualified ARM ID to identity importers (#3203, resolves #3195)
• Fix for asoctl aborting import when an error occurs (#3212, resolves #3200)

Documentation

• Document TIMEOUT variable for testing (#3178)
• Create a landing page for our user guide (#3215, resolves #3215)
• Document CLI differences per shell (#3207, resolves #3142 and #3145)

New Contributors

• @tongpu made their first contribution in #3164

Full Changelog: v2.2.0...v2.3.0

v2.2.0

Release notes

Breaking changes

AKS ManagedClusterServicePrincipalProfile.Secret changed from string to genruntime.SecretReference (#3026)

We always try to avoid breaking changes, but in this case, allowing raw passwords in the spec is a security problem and as such we've
decided to take a break to correct this issue.

Action required: If the ContainerService/ManagedClusters resource is installed in your cluster and the ManagedClusterServicePrincipalProfile.Secret property is set on your ManagedCluster resource, follow the steps in breaking changes document.

Removed the following Status properties, which were never populated (#3034):

• MachineLearningServices:

  • UserAccountCredentials_STATUS.AdminUserPassword
  • UserAccountCredentials_STATUS.AdminUserSshPublicKey
  • VirtualMachineSshCredentials_STATUS.Password

• Synapse:

  • Workspace_STATUS.SqlAdministratorLoginPassword

Upcoming Breaking changes

Beta CRD deprecation

Beta CRD versions (any version with v1beta prefix) will be deprecated no sooner than v2.3.0. We recommend you start using v1api prefixed versions now. You can easily swap from a v1beta version to a v1api version by just replacing v1beta with v1api in your CRD YAML.

Tools

• Tolerate some errors during asoctl import (#3151)
• Fix asoctl when importing resources with fixed names (#3099)

New resources

• DataProtection/BackupVaults (#3078)
• Devices/IotHub (#2999)
• Network/DnsResolver, Network/DnsResolvers/InboundEndpoint and Network/DnsResolvers/OutboundEndpoint (#3046)
• Network/DnsForwardingRuleSet and Network/DnsForwardingRuleSet/ForwardingRule (#3046)
• ContainerService/ManagedCluster/TrustedAccessRoleBinding

Features

• Improve pod securityContext parameters (#3072)
• Export API keys(AdminPrimaryKey, AdminSecondaryKey, QueryKey) for Search/SearchService (#3065)
• Support autogenerating RoleAssignment GUID for AzureName (#3094)
• Export FederatedIdentityCredential.Issuer and FederatedIdentityCredential.Subject as ConfigMaps (#3125)
• Bump cert-manager version to v1.12.1 (#3073)
• Bump controller-runtime version to 0.15.0 (#3138)

Bug Fixes

• Prevent resource drift that could occur without correction by ASO for Resource Providers which work more as a PATCH than a PUT (#3060)
• Resource with reconcile-policy: skip now populates ConfigMap (#2985)
• Fix bug where pre-upgrade check could mistakenly check CRDs that weren't ASO CRDs, causing upgrade to fail (#3128)
• SecurityRules are now merged with NetworkSecurityGroup to avoid clearing and re-create them during reconciliation (#3121)
• Fix networking resources deletion of child resources during adoption (#3136)

Documentation

• Improve our documentation for Dev Setup (#3041)
• Include hand-crafted resources in documentation indexes (#3055)
• Update Managed Identity documentation (#3071)
• Add CRD pattern docs for each group (#3147)

External Contributors

• @shay-ag made their first contribution in #3078
• @khareyash05 made a contribution to increase test coverage in #3134

Full Changelog: v2.1.0...v2.2.0