Skip to content

Latest commit

 

History

History
451 lines (383 loc) · 14.2 KB

README.md

File metadata and controls

451 lines (383 loc) · 14.2 KB
Raw

platform-controllers

Kubernetes first, multi-tenant infrastructure provisioning services

platform.totalsoft.ro

Custom platform resources

Platform

Definition can be found here Example:

apiVersion: platform.totalsoft.ro/v1alpha1
kind: Platform
metadata:
  name: charismaonline.qa
spec:
  code: qa

Tenant

Definition can be found here Example:

apiVersion: platform.totalsoft.ro/v1alpha1
kind: Tenant
metadata:
  name: tenant1
  namespace: qa
spec:
  code: tenant1
  configs:
    config1: value1
    config2: value2
  id: cbb451f2-a7c0-430a-949c-54d576c77b8d
  platformRef: charismaonline.qa

provisioning.totalsoft.ro

monitors infrastructure manifests and provisions the desired infrastructure for every platform / tenant.

Env

Variable example value details
AZURE_LOCATION West Europe default location used to deploy resources in azure
VAULT_ADDR http://localhost:8200 address to vault server
VAULT_TOKEN {token} vault token

Target

The infrastructure manifests can specify wheather the infrastructure can be provisioned for each tenant in a platform, or for the entire platform

Tenant target (default)

The provisioning target can be set to Tenant, allowing provioning of 'per tenant' infrastructure resources.

Note You can skip provisioning for a list of tenants you can specify a Blacklist filter. To allow provisioning for a subset of the tenants, you can specify a Whitelist filter.

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
spec:
  ....
  target:
   category: Tenant
   filter:
     kind: Blacklist
     values:
       - mbfs
       - bnpro

Platform target

The Platform target allows provisioning shared resources for the entire platform.

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
metadata:
  name: mercury-db  
spec:
 ...
  target:
    category: Platform

Dependencies

An infrastructure manifests can specify wheather the provisioned resources depend on the resources provisioned by another manifest. If a dependency is specified, the provisioning of the dependent resource is delayed until the dependency is provisioned.

Note Dependencies can be specified between resources provisioned for the same platform, tenant and domain.

The dependency list is specified in the "dependsOn" present on every provisoning manifest. A dependency is identified by kind and name.

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureManagedDatabase
metadata:
  name: my-db
spec:
  ...

---

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: HelmRelease
metadata:
  name: my-server
spec:
  dependsOn:
    - kind: AzureManagedDatabase
      name: my-db
  ...

AzureDatabase

Definition can be found here

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureDatabase
spec:
  dbName: origination_db
  domainRef: origination
  exports:
    - domain: origination
      dbName:
        toConfigMap:
          keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ConnectionStrings__Database
  platformRef: charismaonline.qa
  sourceDatabaseId: /subscriptions/XXXXXX-b8a5-4967-a05a-2fa3c4295710/resourceGroups/SQLMI_RG/providers/Microsoft.Sql/servers/r7ddbsrv/databases/insurance_db
  sqlServer:
    elasticPoolName: dbpool
    resourceGroupName: SQL_RG
    serverName: r7ddbsrv

Note You can skip provisioning for some tenant by adding the label provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"

AzureManagedDatabase

Definition can be found here

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureManagedDatabase
spec:
  dbName: origination_db
  domainRef: origination
  exports:
    - domain: origination
      dbName:
        toConfigMap:
          keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ConnectionStrings__Database
  managedInstance:
    name: incubsqlmi
    resourceGroup: SQLMI_RG
  restoreFrom:
    backupFileName: origination_test_backup_2022_04_13_154723.bak
    storageContainer:
      uri: https://my-blobstorage.blob.core.windows.net/backup-repository
      sasToken: my-saas-token
  platformRef: charismaonline.qa

Note You can skip provisioning for some tenant by adding the label provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"

HelmRelease

Definition can be found here

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: HelmRelease
metadata:
  name: my-helm-release
  namespace: my-namespace
spec:
  domainRef: origination
  platformRef: radu.demo
  release:
    interval: 10m
    releaseName: my-release
    chart:
      spec:
        version: ">=0.1.0-0"
        chart: my-chart
        sourceRef:
          kind: HelmRepository
          name: my-helm-repo
          namespace: my-namespace
    upgrade:
      remediation:
        remediateLastFailure: false
    values:
      global:
        vaultEnvironment: "false"

        runtimeConfiguration:
          enabled: false
          configMap: origination-aggregate
          csi:
            secretProviderClass: origination-aggregate

Note You can skip provisioning for some tenant by adding the label provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"

AzureVirtualMachine

Definition can be found here

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureVirtualMachine
metadata:
  name: charisma-client-vm
  namespace: qa-lsng
spec:
  domainRef: origination
  enableTrustedLaunch: false
  exports:
    - domain: origination
      adminPassword:
        toVault:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_Admin_Password
      adminUserName:
        toVault:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_Admin_UserName
      computerName:
        toConfigMap:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_ComputerName      
      publicAddress:
        toConfigMap:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_PublicAddress
      vmName:
        toConfigMap:
          keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__CharismaClient_VM_Name
  osDiskType: Standard_LRS
  platformRef: charismaonline.qa
  rdpSourceAddressPrefix: 128.0.57.0/25
  sourceImageId: >-
    /subscriptions/05a50a12-6628-4627-bd30-19932dac39f8/resourceGroups/Provisioning_Test/providers/Microsoft.Compute/galleries/MyGallery/images/ch-client-base/versions/2.0.0
  subnetId: >-
    /subscriptions/05a50a12-6628-4627-bd30-19932dac39f8/resourceGroups/charismaonline.qa/providers/Microsoft.Network/virtualNetworks/charismaonline-vnet/subnets/default
  vmName: charisma-client
  vmSize: Standard_B1s

Note You can skip provisioning for some tenant by adding the label provisioning.totalsoft.ro/skip-tenant-SOME_TENANT_CODE="true"

AzureVirtualDesktop

Definition can be found here

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzureVirtualDesktop
metadata:
  name: charisma-client-avd
  namespace: qa-lsng
spec:
  applications:
    - friendlyName: Charisma Enterprise
      name: charisma-enterprise
      path: >-
        C:\Program Files (x86)\TotalSoft\Charisma Enterprise\Windows Client\Charisma.WinUI.exe
  domainRef: origination
  enableTrustedLaunch: false
  exports:
    - adminPassword:
        toVault:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code
            }}__CharismaClient_Admin_Password
      adminUserName:
        toVault:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code
            }}__CharismaClient_Admin_UserName
      computerName:
        toConfigMap:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code
            }}__CharismaClient_ComputerName
      domain: origination
      hostPoolName:
        toConfigMap:
          keyTemplate: >-
            MultiTenancy__Tenants__{{ .Tenant.Code
            }}__CharismaClient_HostPool_Name
  hostPoolName: ch-client-fs
  initScript: |
    param
    (    
      [Parameter(Mandatory = $true)]
      [String]$ChServerIP
    )

    Write-Host $ChServerIP
  initScriptArgs:
    - name: ChServerIP
      value: 11.12.13.17
  osDiskType: Premium_LRS
  platformRef: charismaonline.qa
  sourceImageId: >-
    /subscriptions/15b38e46-ef41-4f5b-bdba-7d9354568c2d/resourceGroups/test-vm/providers/Microsoft.Compute/galleries/LFGalery/images/base-avd/versions/1.0.0
  subnetId: >-
    /subscriptions/15b38e46-ef41-4f5b-bdba-7d9354568c2d/resourceGroups/test-vm/providers/Microsoft.Network/virtualNetworks/ch-client-base-vnet/subnets/default
  users:
    admins:
      - admin-mbfs@test.onmicrosoft.com
    applicationUsers:
      - user1-mbfs@test.onmicrosoft.com
  vmNamePrefix: ch-client-fs
  vmNumberOfInstances: 2
  vmSize: Standard_B2s
  workspaceFriendlyName: Charisma

AzurePowershellScript

AzurePowershellScript is a Custom Resource Definition (CRD) that represents an Azure PowerShell deployment script.

Definition can be found here

Spec

The AzurePowershellScript spec has the following fields:

  • scriptContent: The content of the PowerShell script to be executed.
  • scriptArguments: The arguments to be passed to the PowerShell script. These should match the parameters defined in the scriptContent.
  • domainRef: The reference to the domain that the user belongs to.
  • platformRef: The reference to the platform that the user belongs to.
  • forceUpdateTag: Update this value to trigger the script even if the content or args are unchanged

Example:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: AzurePowerShellScript
metadata:
  name: createresourcegroup
  namespace: provisioning-test
spec:
  domainRef: domain2
  exports:
    - scriptOutputs:
        toConfigMap:
          keyTemplate: MultiTenancy__Tenants__{{ .Tenant.Code }}__ScriptOutputs  
  platformRef: provisioning.test
  scriptContent: |-
    param([string] $name)

    $output = "RG name: {0}" -f $name 
    Write-Output $output  

    $DeploymentScriptOutputs = @{} 
    $DeploymentScriptOutputs['text'] = $output

    New-AzResourceGroup $name "West Europe"
  scriptArguments: "-name testrg-{{ .Platform }}-{{ .Tenant.Code }}"
  target:
    category: Tenant

EntraUser

EntraUser is a Custom Resource Definition (CRD) that represents a user for Entra Id.

Definition can be found here

Spec

The EntraUser spec has the following fields:

  • userPrincipalName: The user principal name of the user. This is typically the user's email address or username.
  • displayName: The display name of the user.
  • initialPassword: The initial password for the user. If this is not provided, a random password will be generated.
  • domainRef: The reference to the domain that the user belongs to.
  • platformRef: The reference to the platform that the user belongs to.

Example

Here's an example of an EntraUser resource:

apiVersion: provisioning.totalsoft.ro/v1alpha1
kind: EntraUser
metadata:
  name: example-user
  namespace: qa-lsng
spec:
  userPrincipalName: "user@tenant1-qa.example.com"
  displayName: "Example User"
  initialPassword: "password123"
  domainRef: "entra-users"
  platformRef: "qa"
  exports:
    - domain: entra-users
      initialPassword:
        toVault:
          keyTemplate: InitialPassword
      userPrincipalName:
        toVault:
          keyTemplate: UserPrincipalName

configuration.totalsoft.ro

manages external configuration for the services in the platform, read more about from the Twelve-Factor App methodology.

ConfigurationDomain

Definition can be found here

If aggregateConfigMaps is set, it will aggregate all config maps from it's namespace and platform's target namespace, for the specified domain and generates an output config map in the same namespace.

If aggregateSecrets is set, it will aggregate all secrets stored in vault for the specified platform namespace and domain and generates an output CSI SecretProviderClass namespace.

Example:

apiVersion: configuration.totalsoft.ro/v1alpha1
kind: ConfigurationDomain
metadata:
  name: origination
  namespace: qa-lsng
spec:
  aggregateConfigMaps: true
  aggregateSecrets: true
  platformRef: charismaonline.qa

Note 1 The monitored config maps can be either in the same namespace as the ConfigurationDomain or in the platform's target namespace, they are identified by platform.totalsoft.ro/domainand platform.totalsoft.ro/platform labels.

Note 2 There is support for global platform config maps, in this case the platform.totalsoft.ro/domain label has the value "global". These global config maps are always monitored and aggregated with the current domain config maps.

Note 3 The monitored vault secrets are organized by platform (secret engine), namespace (subfolder) and domain (subfolder). The keys of the secrets should be the environment variable names, and if they are not unique for the domain they will be overwritten.

Note 4 There is support for global platform secrets (subfolder) and global namespace secrets (subfolder), in this case the domain folder should be named "global". These global secrets are always monitored and aggregated with the current domain secrets.

For example, for the above manifest, the controller will aggregate secrets from the following paths:

  • /charismaonline.qa/qa/global
  • /charismaonline.qa/qa-lsng/global
  • /charismaonline.qa/qa-lsng/origination/