Include system affinity for when ASO runs on AKS

[theunrepentantgeek]

Describe the improvement

A suggestion made via email:

My suggestion to improve ASO would be to include the following node affinity block that would target system nodepools on any AKS cluster:

affinity:
    nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.azure.com/mode
              operator: In
              values:
                - system

According to @matthchr we shouldn't use required because it ASO is supported on non AKS clusters, but there's a preferredDuringSchedulingIgnoredDuringExecution we could use too.

[matthchr]

What do we have set for mem/cpu requirements - are they too low?

[matthchr]

We may need a negative assertion rather than positive: "Don't schedule on the user pool", rather than "must schedule on the system pool"

and wahtever we do should probably be put in a Helm "if" block.

[theunrepentantgeek]

We've discussed this internally and we don't think adding an AKS specific affinity is appropriate, given we want ASO to run well on other kinds of cluster. We will add documentation to make this easier to find, and if/when AKS supports ASO as an extension we'll do this by default.