GPU Type #419

wild-endeavor · 2023-06-27T19:08:26Z

TL;DR

Adds support for passing along extra metadata related to resources (e.g. accelerator type) to the backend

Type

Bug Fix
Feature
Plugin

Are all requirements met?

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

Remove the 'fixes' keyword if there will be multiple PRs to fix the linked issue

fixes https://github.com/flyteorg/flyte/issues/

Follow-up issue

NA
OR
https://github.com/flyteorg/flyte/issues/

codecov · 2023-06-27T20:00:30Z

Codecov Report

Patch coverage has no change and project coverage change: +2.55% 🎉

Comparison is base (7f62a15) 75.92% compared to head (fe70c3d) 78.48%.

❗ Current head fe70c3d differs from pull request most recent head 152a861. Consider uploading reports for the commit 152a861 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #419      +/-   ##
==========================================
+ Coverage   75.92%   78.48%   +2.55%     
==========================================
  Files          18       18              
  Lines        1458     1250     -208     
==========================================
- Hits         1107      981     -126     
+ Misses        294      212      -82     
  Partials       57       57

Flag	Coverage Δ
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 18 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Pass task execution metadata from agent Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Add doc Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Update protos/flyteidl/admin/agent.proto Co-authored-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> * Regenerate --------- Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> Co-authored-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add comment Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Daniel Rammer <daniel@union.ai> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

EngHabu · 2023-09-21T01:25:28Z

protos/flyteidl/core/tasks.proto

+
+// Additional metadata associated with resources to allocate to a task
+message ResourceMetadata {
+    oneof accelerator_value {


Why oneof? if it's not set, it'll just be nil, do you want to allow nullifying a field? if so maybe that should go into the override message?

The thought was to support other accelerator types like TPUs in the future. One option would have been to just add all possible accelerator types as separate fields without the guarantee that oneof provides - that only one type is set at any given point in time.

I actually did not consider nullifying an accelerator specified in the task metadata, in the override. That does sound nice. If we were to do that, would it be a bad idea to add a accelerator_not_set to the oneof?

Since oneofs do not change wire format, I would keep it out and kick the can down the road when we add a different accelerator to decide then.

There isn't a great way to nullify in Protos unfortunately...

This reverts commit ef37788.

This reverts commit ef37788. Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* Revert "GPU Type (#419)" This reverts commit ef37788. Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Restore .readthedocs.yml Signed-off-by: Jeev B <jeevb@users.noreply.github.com> --------- Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

* add field Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Pass task execution metadata from agent (#422) * Pass task execution metadata from agent Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Add doc Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> * Update protos/flyteidl/admin/agent.proto Co-authored-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> * Regenerate --------- Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> Co-authored-by: Kevin Su <pingsutw@gmail.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Add tags to execution spec (#414) * add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add tags to execution spec Signed-off-by: Kevin Su <pingsutw@apache.org> * add comment Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Correct comment for array job max parallelism (#431) Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Add the scalar to the operand (#427) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * add selector Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * move selectors from container to task metadata Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * drop only_preferred Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Updating boilerplate to lock golangci-lint version (#435) Signed-off-by: Daniel Rammer <daniel@union.ai> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * add unpartitioned selector Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * refactor Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * refactor Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * fix oneof names Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * add build.os for read the docs Signed-off-by: Jeev B <jeevb@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com> Signed-off-by: Hongxin Liang <honnix@users.noreply.github.com> Signed-off-by: Honnix <honnix@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Daniel Rammer <daniel@union.ai> Co-authored-by: Honnix <honnix@users.noreply.github.com> Co-authored-by: Kevin Su <pingsutw@gmail.com> Co-authored-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Katrina Rogan <katroganGH@gmail.com> Co-authored-by: Jeev B <jeevb@users.noreply.github.com> Co-authored-by: Dan Rammer <daniel@union.ai>

* Revert "GPU Type (#419)" This reverts commit 7bd98a9. Signed-off-by: Jeev B <jeevb@users.noreply.github.com> * Restore .readthedocs.yml Signed-off-by: Jeev B <jeevb@users.noreply.github.com> --------- Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

jeevb force-pushed the gpu-type branch from c52b42f to 35bd333 Compare September 8, 2023 19:37

wild-endeavor and others added 13 commits September 18, 2023 08:51

add field

8a3155f

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Correct comment for array job max parallelism (#431)

39a0876

Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Add the scalar to the operand (#427)

df35fb2

Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

add selector

f622b94

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

move selectors from container to task metadata

713b81f

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

drop only_preferred

eeb1130

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Updating boilerplate to lock golangci-lint version (#435)

e5980b9

Signed-off-by: Daniel Rammer <daniel@union.ai> Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

add unpartitioned selector

28ebaa3

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

refactor

a8f8d48

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

refactor

ac7df01

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

fix oneof names

8e46a8c

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

jeevb force-pushed the gpu-type branch from dc07c49 to 8e46a8c Compare September 18, 2023 15:51

Merge branch 'master' into gpu-type

d6ac468

jeevb marked this pull request as ready for review September 19, 2023 19:41

hamersaw approved these changes Sep 20, 2023

View reviewed changes

jeevb force-pushed the gpu-type branch from d8e86a5 to 9623156 Compare September 20, 2023 18:52

add build.os for read the docs

152a861

Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

jeevb force-pushed the gpu-type branch from 9623156 to 152a861 Compare September 20, 2023 18:53

jeevb enabled auto-merge (squash) September 20, 2023 20:05

pingsutw approved these changes Sep 20, 2023

View reviewed changes

jeevb merged commit ef37788 into master Sep 20, 2023
13 checks passed

jeevb deleted the gpu-type branch September 20, 2023 20:11

jeevb mentioned this pull request Sep 20, 2023

[Extended Resources] GPU Accelerators flyteorg/flytekit#1843

Merged

8 tasks

EngHabu reviewed Sep 21, 2023

View reviewed changes

jeevb mentioned this pull request Sep 21, 2023

Remove accelerator_value oneof in core.ResourceMetadata #442

Closed

jeevb added a commit that referenced this pull request Sep 22, 2023

Revert "GPU Type (#419)"

c9d02c8

This reverts commit ef37788.

jeevb added a commit that referenced this pull request Sep 22, 2023

Revert "GPU Type (#419)"

e89c2c9

This reverts commit ef37788. Signed-off-by: Jeev B <jeevb@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GPU Type #419

GPU Type #419

wild-endeavor commented Jun 27, 2023 •

edited by jeevb

Loading

codecov bot commented Jun 27, 2023 •

edited

Loading

EngHabu Sep 21, 2023

jeevb Sep 21, 2023 •

edited

Loading

EngHabu Sep 21, 2023

jeevb Sep 21, 2023

GPU Type #419

GPU Type #419

Conversation

wild-endeavor commented Jun 27, 2023 • edited by jeevb Loading

TL;DR

Type

Are all requirements met?

Complete description

Tracking Issue

Follow-up issue

codecov bot commented Jun 27, 2023 • edited Loading

Codecov Report

EngHabu Sep 21, 2023

Choose a reason for hiding this comment

jeevb Sep 21, 2023 • edited Loading

Choose a reason for hiding this comment

EngHabu Sep 21, 2023

Choose a reason for hiding this comment

jeevb Sep 21, 2023

Choose a reason for hiding this comment

wild-endeavor commented Jun 27, 2023 •

edited by jeevb

Loading

codecov bot commented Jun 27, 2023 •

edited

Loading

jeevb Sep 21, 2023 •

edited

Loading