Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Argo Workflow into ODAHU for GKE #77

Closed
karbyshevds opened this issue Feb 22, 2021 · 3 comments · Fixed by #123
Closed

Add Argo Workflow into ODAHU for GKE #77

karbyshevds opened this issue Feb 22, 2021 · 3 comments · Fixed by #123
Assignees
@easokol
Labels
1.5

Comments

@karbyshevds
Copy link
Collaborator

karbyshevds commented Feb 22, 2021 •
edited by easokol
Loading

Automate Argo Workflow](https://argoproj.github.io/) deployment into ODAHU gke cluster so it can be used for ML pipelines orchestration on top of ODAHU APIs.
@easokol easokol changed the title Argo CD integration Argo Workflows integration Mar 3, 2021
@easokol easokol self-assigned this Mar 3, 2021
karbyshevds pushed a commit that referenced this issue Apr 3, 2021
[#77] Fix firewall cleanup script call
7a72e39
karbyshevds pushed a commit that referenced this issue Apr 3, 2021
[#77] Optionally install Argo Workflows to GKE
5c8100c
karbyshevds pushed a commit that referenced this issue Apr 5, 2021
[#77] Add GCS artifact storage to GKE Argo installation
8e274a4
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Use separate workflows namespace if provided
7cedf1d
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Add artifact storage params
b3beecb
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Use single namespace
4dffefc
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Cleanup
b9f60e8
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Use default artifact repo
c87fdb8
@dsuslov dsuslov changed the title Argo Workflows integration Add Argo into ODAHU Apr 6, 2021
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Fix argo target bucket
65ac19b
@dsuslov dsuslov changed the title Add Argo into ODAHU Add Argo Workflow into ODAHU Apr 6, 2021
@dsuslov dsuslov assigned karbyshevds and unassigned easokol Apr 6, 2021
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Fix argo target bucket
5c6b7b0
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Fix argo sa annotations
9704d50
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Fix argo sa annotations
147bc4c
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Argo: Fix SA permissions
87d0bdc
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Argo: Fix SA permissions
d2571a4
karbyshevds pushed a commit that referenced this issue Apr 6, 2021
[#77] Argo: Fix SA permissions
b1c88ab
karbyshevds pushed a commit that referenced this issue Apr 7, 2021
[#77] Fix artifacts access from Argo UI
7fc0ee6
karbyshevds pushed a commit that referenced this issue Apr 7, 2021
[#77] Add annotations to server SA
fc73ec2
karbyshevds pushed a commit that referenced this issue Apr 7, 2021
[#77] Add annotations to server SA
b7322c0
@karbyshevds
Copy link
Collaborator Author

The major issues faced:

  • s3: no KMS support, see this issue.
  • got CSS errors in browser console, ticket is here.
  • There is a bunch or annoying issues when Argo works behind the proxy, e.x. this one.

karbyshevds pushed a commit that referenced this issue Apr 12, 2021
[#77] revert terragrunt.hcl changes
f7632b3
karbyshevds pushed a commit that referenced this issue Apr 12, 2021
[#77] remove Argo from default external links
f4e16f4
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Use default artifact repo
8301b4a
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Fix argo target bucket
c2eed49
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Fix argo target bucket
1c3ba9b
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Fix argo sa annotations
8cb54a8
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Fix argo sa annotations
775f8d8
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: Fix SA permissions
739e0e8
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: Fix SA permissions
9ae5446
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: Fix SA permissions
ffd50fb
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Fix artifacts access from Argo UI
cf91643
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Add annotations to server SA
85daa02
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Add annotations to server SA
998dfb8
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] revert terragrunt.hcl changes
a4cb027
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] remove Argo from default external links
de232b3
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: main module name change main->helm
468bfed
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77]argo: parametrize some some stuff for GKE
dceec04
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: parametrize Argo version, selectors & taints
aa1c153
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: EKS WiP
34c64fb
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: EKS prereqs
bc83bf6
karbyshevds pushed a commit that referenced this issue Apr 13, 2021
[#77] Argo: add Minio module [WiP]
e688c9c
@karbyshevds karbyshevds assigned easokol and unassigned karbyshevds Apr 13, 2021
@karbyshevds
Copy link
Collaborator Author

As Minio required for AWS Argo Workflows installation, #17 is also affected.

@karbyshevds
Copy link
Collaborator Author

After Minio integration, s3 credentials should be passed to Argo module. Inside the module useStaticCredentials should be true, see this values

@easokol easokol changed the title Add Argo Workflow into ODAHU Add Argo Workflow into ODAHU for GKE May 25, 2021
@easokol easokol mentioned this issue May 28, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@easokol easokol

Labels
1.5
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Argo Workflow deployment odahu/odahu-automation
2 participants
@karbyshevds @easokol