Support users to set more fields in suggetion deployment

[tingweiwu]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

Is it possible to allow users to set more fields in suggetion deployment like we set the trailSpec
such as imagePullSecrets 、args、command and so on

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

[gaocegege]

I think it is a useful feature. WDYT @kubeflow/wg-automl-leads

[andreyvelich]

Thank you for creating this @tingweiwu!
We can allow to specify the Deployment spec or spec.template.spec in Config Suggestion Settings, but then setting just Suggestion image will be more complicated for our users since they have to follow the correct API structure.
For example:

"random": {
  "containers": [
    {
      "image": "docker.io/andreyvelichkevich/suggestion-chocolate:latest"
    }
  ]
 },

Does it look good to us @gaocegege @johnugeorge @tingweiwu ?

[tingweiwu]

It seems the Suggestion settings with configmap don't support imagePullSecrets and other similar spec settings in Deployment

It will probably be more flexible for user to set deployment attributes in algorithmSpec template in different scenarios.
two simple examples as the following
case 1: set imagePullSecrets and resource limits

apiVersion: kubeflow.org/v1beta1
kind: Experiment
metadata:
  namespace: kubeflow-test
  name: job-test
spec:
    algorithm:
      algorithmName: random
      algorithmSpec:
        template:
          spec:
            containers:
            - image: docker.io/kubeflowkatib/suggestion-hyperopt:latest
              resources:
                limits:
                  cpu: 500m
                  ephemeral-storage: 5Gi
                  memory: 100Mi
                requests:
                  cpu: 50m
                  ephemeral-storage: 500Mi
                  memory: 10Mi 
            imagePullSecrets:
            - name: default-secret  
                 

case 2: set start commond and persistent volume, as the new suggetion service may not build in image.

apiVersion: kubeflow.org/v1beta1
kind: Experiment
metadata:
  namespace: kubeflow-test
  name: job-test
spec:
    algorithm:
      algorithmName: random
      algorithmSpec:
        template:
          spec:
            containers:
            - image: docker.io/kubeflowkatib/suggestion-new:latest
              args:
              - python /opt/apps/main.py 
              command:
              - /bin/bash
              - -c
              volumeMounts:
                - mountPath: /opt/apps
                  name: kubeflow-test-pvc
                  readOnly: true
            imagePullSecrets:
            - name: default-secret  
            volumes:
            - name: kubeflow-test-pvc
              persistentVolumeClaim:
                claimName: kubeflow-test-pvc

[gaocegege]

I think we should consider if we can support SuggestionSpec as @tingweiwu said. WDYT @andreyvelich

[andreyvelich]

Basically, Katib config was designed to simplify Experiment specification and give additional settings to more advanced users who are Kubernetes experts.
Therefore, I am not sure that algorithmSpec should live under Experiment, but we can extend Katib config settings as I mentioned above: #1737 (comment).

What do you think @johnugeorge ?

[andreyvelich]

/priority p2
/help

[google-oss-prow]

@andreyvelich:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/priority p2
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[johnugeorge]

I think, this is a valid ask for many deployments. We can provide this as an option. If AlgorithmSpec is provided in Experiment, it is picked up. Else, defaults will be used(like the current)

[ramdootp]

@andreyvelich @johnugeorge would need this feature for our deployments. which katib release will have this feature implemented?

[andreyvelich]

This hasn't been implemented fully yet.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[andreyvelich]

/lifecycle frozen