[Question] S3 Artifact: Handling Multiple Accounts & Multiple Buckets

[naiduasn]

Is this a BUG REPORT or FEATURE REQUEST?:
Question
What happened:
I want to manage below use cases via workflow controller config

1. All output artifacts will be available in one bucket
2. All logs will be available in one bucket
3. All temp artifacts like git will be in one bucket with expiration policy set to auto delete

How can i configure these at a global argo level?

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Argo version:

$ argo version

• Kubernetes version :

$ kubectl version -o yaml

Other debugging information (if applicable):

• workflow result:

$ argo get <workflowname>

• executor logs:

$ kubectl logs <failedpodname> -c init
$ kubectl logs <failedpodname> -c wait

• workflow-controller logs:

$ kubectl logs -n argo $(kubectl get pods -l app=workflow-controller -n argo -o name)

[naiduasn]

any way i can achieve this? @simster7 @sarabala1979

[sarabala1979]

if you want to achieve the above use-case on workflow-controller-configmap(centralized), currently there is no support for multi S3 buckets in config. we can enhance to support multiple ArtifactRepositories config in configmap and it will be referred in workflow input and output.

[naiduasn]

@sarabala1979 Yes, Enhancement to support Multiple Artifact repositories can allow me to create a workaround.

[alexec]

See #2157

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.