Skip to content

Latest commit

 

History

History
176 lines (120 loc) · 11.4 KB

deploy.md

File metadata and controls

176 lines (120 loc) · 11.4 KB
Raw

Deploy a Kubernetes Cluster

Install Prerequisites

All the commands in this guide require both the Azure CLI and aks-engine. Follow the installation instructions to download aks-engine before continuing or compile from source.

For installation instructions see the Azure CLI GitHub repository for the latest release.

Overview

aks-engine reads a cluster definition which describes the size, shape, and configuration of your cluster. This guide takes the default configuration of one master and two Linux agents. If you would like to change the configuration, edit examples/kubernetes.json before continuing.

The aks-engine deploy command automates creation of a Service Principal, Resource Group and SSH key for your cluster. If operators need more control or are interested in the individual steps see the "Long Way" section below.

NOTE: AKS Engine creates a cluster; it doesn't create an Azure Container Service resource. So clusters that you create using the aks-engine command (or ARM templates generated by the aks-engine command) won't show up as AKS resources, for example when you run az acs list. Think of aks-engine as the, er, engine which AKS uses to create clusters: you can use the same engine yourself, but AKS won't know about the results.

After the cluster is deployed the upgrade and scale commands can be used to make updates to your cluster.

Gather Information

  • The subscription in which you would like to provision the cluster. This is a uuid which can be found with az account list -o table.
  • Proper access rights within the subscription. Especially the right to create and assign service principals to applications ( see AKS Engine the Long Way, Step #2)
  • A valid service principal with all the required create/manage permissions. Instructions to create a new service principal can be found here.
  • A dnsPrefix which forms part of the the hostname for your cluster (e.g. staging, prodwest, blueberry). The DNS prefix must be unique so pick a random name.
  • A location to provision the cluster e.g. westus2.
$ az account list -o table
Name                                             CloudName    SubscriptionId                        State    IsDefault
-----------------------------------------------  -----------  ------------------------------------  -------  -----------
Contoso Subscription                             AzureCloud   51ac25de-afdg-9201-d923-8d8e8e8e8e8e  Enabled  True

Deploy

For this example, the subscription id is 51ac25de-afdg-9201-d923-8d8e8e8e8e8e, the DNS prefix is contoso-apple, and location is westus2.

Run aks-engine deploy with the appropriate arguments:

$ aks-engine deploy --subscription-id 51ac25de-afdg-9201-d923-8d8e8e8e8e8e \
    --client-id '<service principal client ID>' \
    --client-secret '<service principal client secret>' \
    --dns-prefix contoso-apple --location westus2 \
    --api-model examples/kubernetes.json

WARN[0005] apimodel: missing masterProfile.dnsPrefix will use "contoso-apple"
WARN[0005] --resource-group was not specified. Using the DNS prefix from the apimodel as the resource group name: contoso-apple
INFO[0034] Starting ARM Deployment (contoso-apple-1423145182). This will take some time...
INFO[0393] Finished ARM Deployment (contoso-apple-1423145182).

aks-engine will output Azure Resource Manager (ARM) templates, SSH keys, and a kubeconfig file in _output/contoso-apple-59769a59 directory:

  • _output/contoso-apple-59769a59/azureuser_rsa
  • _output/contoso-apple-59769a59/kubeconfig/kubeconfig.westus2.json

aks-engine generates kubeconfig files for each possible region. Access the new cluster by using the kubeconfig generated for the cluster's location. This example used westus2, so the kubeconfig is _output/<clustername>/kubeconfig/kubeconfig.westus2.json:

$ KUBECONFIG=_output/contoso-apple-59769a59/kubeconfig/kubeconfig.westus2.json kubectl cluster-info
Kubernetes master is running at https://contoso-apple-59769a59.westus2.cloudapp.azure.com
Heapster is running at https://contoso-apple-59769a59.westus2.cloudapp.azure.com/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at https://contoso-apple-59769a59.westus2.cloudapp.azure.com/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://contoso-apple-59769a59.westus2.cloudapp.azure.com/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Administrative note: By default, the directory where aks-engine stores cluster configuration (_output/contoso-apple above) won't be overwritten as a result of subsequent attempts to deploy a cluster using the same --dns-prefix) To re-use the same resource group name repeatedly, include the --force-overwrite command line option with your aks-engine deploy command. On a related note, include an --auto-suffix option to append a randomly generated suffix to the dns-prefix to form the resource group name, for example if your workflow requires a common prefix across multiple cluster deployments. Using the --auto-suffix pattern appends a compressed timestamp to ensure a unique cluster name (and thus ensure that each deployment's configuration artifacts will be stored locally under a discrete _output/<resource-group-name>/ directory).

Note: If the cluster is using an existing VNET please see the Custom VNET feature documentation for additional steps that must be completed after cluster provisioning.

The deploy command lets you override any values under the properties tag (even in arrays) from the cluster definition file without having to update the file. You can use the --set flag to do that. For example:

aks-engine deploy --resource-group "your-resource-group" \
  --location "westeurope" \
  --subscription-id "your-subscription-id" \
  --client-id '<your service principal client ID>' \
  --client-secret '<your service principal client secret>' \
  --api-model "./apimodel.json" \
  --set masterProfile.dnsPrefix="your-dns-prefix-override" \
  --set agentPoolProfiles[0].name="your-agentpool-0-name-override" \
  --set agentPoolProfiles[0].count=1 \
  --set linuxProfile.ssh.publicKeys[0].keyData="ssh-rsa PUBLICKEY azureuser@linuxvm" \
  --set servicePrincipalProfile.clientId="spn-client-id" \
  --set servicePrincipalProfile.secret="spn-client-secret"

AKS Engine the Long Way

Step 1: Generate an SSH Key

In addition to using Kubernetes APIs to interact with the clusters, cluster operators may access the master and agent machines using SSH.

If you don't have an SSH key cluster operators may generate a new one.

Step 2: Create a Service Principal

Kubernetes clusters have integrated support for various cloud providers as core functionality. On Azure, aks-engine uses a Service Principal to interact with Azure Resource Manager (ARM). Follow the instructions to create a new service principal and grant it the necessary IAM role to create Azure resources.

Step 3: Edit your Cluster Definition

AKS Engine consumes a cluster definition which outlines the desired shape, size, and configuration of Kubernetes. There are a number of features that can be enabled through the cluster definition: check the examples directory for a number of... examples.

Edit the simple Kubernetes cluster definition and fill out the required values:

  • dnsPrefix: must be a region-unique name and will form part of the hostname (e.g. myprod1, staging, leapingllama) - be unique!
  • keyData: must contain the public portion of an SSH key - this will be associated with the adminUsername value found in the same section of the cluster definition (e.g. 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA....')
  • clientId: this is the service principal's appId uuid or name from step 2
  • secret: this is the service principal's password or randomly-generated password from step 2

Optional: attach to an existing virtual network (VNET). Details here

Note: you can then use the --set option of the generate command to override values from the cluster definition file directly in the command line (cf. Step 4)

Step 4: Generate the Templates

The generate command takes a cluster definition and outputs a number of templates which describe your Kubernetes cluster. By default, generate will create a new directory named after your cluster nested in the _output directory. If my dnsPrefix was larry my cluster templates would be found in _output/larry-.

Run aks-engine generate examples/kubernetes.json

The generate command lets you override values from the cluster definition file without having to update the file. You can use the --set flag to do that:

aks-engine generate --set linuxProfile.adminUsername=myNewUsername,masterProfile.count=3 clusterdefinition.json

The --set flag only supports JSON properties under properties. You can also work with array, like the following:

aks-engine generate --set agentPoolProfiles[0].count=5,agentPoolProfiles[1].name=myPoolName clusterdefinition.json

Step 5: Submit your Templates to Azure Resource Manager (ARM)

Deploy the output azuredeploy.json and azuredeploy.parameters.json

  • To enable the optional network policy enforcement using calico, you have to set the parameter during this step according to this guide
  • To enable the optional network policy enforcement using cilium, you have to set the parameter during this step according to this guide
  • To enable the optional network policy enforcement using antrea, you have to set the parameter during this step according to this guide

Note: If the cluster is using an existing VNET please see the Custom VNET feature documentation for additional steps that must be completed after cluster provisioning.

Checking VM tags

First we get list of Master and Agent VMs in the cluster

az vm list -g <resource group of cluster> -o table
Name                      ResourceGroup                    Location
------------------------  -------------------------------  -------------
k8s-agentpool1-22116803-1       XXXXXXXXXXXX                  southeastasia
k8s-master-22116803-0           XXXXXXXXXXXX                  southeastasia

Once we have the VM Names, we can check tags associated with any of the VMs using the command below

az vm show -g <resource group of cluster> -n <name of Master or agent VM> --query tags
Sample JSON out of this command is shown below. This command can also be used to check the aks-engine version which was used to create the cluster

{
  "aksEngineVersion": "v0.35.1",
  "creationSource": "aksengine-k8s-master-22116803-0",
  "orchestrator": "Kubernetes:1.12.8",
  "poolName": "master",
  "resourceNameSuffix": "22116803"
}