Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chain dependencies between multiple promises #2955

Closed
2 tasks done
roek5803 opened this issue Oct 5, 2022 · 1 comment
Closed
2 tasks done

Chain dependencies between multiple promises #2955

roek5803 opened this issue Oct 5, 2022 · 1 comment
Labels
enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers

Comments

@roek5803
Copy link

roek5803 commented Oct 5, 2022

Motivation: Why do you think this is important?

Right now to chain tasks which returns None within a workflow you can do that in multiple ways. One of the ways is with >>, though when doing it you can only chain 2 promises like: promise1 >> promise2, but you can not chain more promises like: promise1 >> promise2 >> promise3 then i get this error: TypeError: unsupported operand type(s) for >>: 'NoneType' and 'VoidPromise'.

Goal: What should the final outcome look like, ideally?

It would be good if you can use this syntax in order to chain multiple (more than 2) promises like this:

from time import sleep

from flytekit import task, workflow


@task
def task_a():
    sleep(4)
    print("task_a")


@task
def task_b():
    sleep(1)
    print("task_b")


@task
def task_c():
    print("task_c")


@workflow
def my_wf():
    promise1 = task_a()
    promise2 = task_b()
    promise3 = task_c()

    promise1 >> promise2 >> promise3


if __name__ == "__main__":
    my_wf()

Also this should be possible:

@workflow
def my_wf():
    task_a() >> task_b() >> task_c()

Describe alternatives you've considered

This is just an improvement of existing functionality.

Propose: Link/Inline OR Additional context

I didn't spend yo much time looking into how it should be fixed, but i guess it would be here: https://github.com/flyteorg/flytekit/blob/0bd3261c7567b0e25fa4cac88c883279b8752447/flytekit/core/promise.py#L350 where the function __rshift__ needs to return other promise

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@roek5803 roek5803 added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers labels Oct 5, 2022
@welcome
Copy link

welcome bot commented Oct 5, 2022

Thank you for opening your first issue here! 🛠

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers
Projects
None yet
Development

No branches or pull requests

2 participants