You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello,
I was trying to debug this for a very long time.. It seems that if you have sub-tasks with the same label they will not run in parallel. For example if you have:
for i in [1,2,3]:
self.addWorkflowTask("task-%d" % i, RunFoo())
class RunFoo(WorkflowRunner):
def workflow(self):
self.addTask("foo", cmd)
self.addTask("bar", cmd)
Only one "foo" and "bar" will run in parallel, even though their labels are "task-1+foo", "task-1+bar", "task-2+foo", "task-2+bar", "task-3+foo", "task-3+bar"
A quick workaround is this:
for i in [1,2,3]:
self.addWorkflowTask("task-%d" % i, RunFoo(pid))
class RunFoo(WorkflowRunner):
def __init__(self, pid):
self.pid = pid
def workflow(self):
self.addTask("foo-%d" % self.pid, cmd)
self.addTask("bar-%d" % self.pid, cmd)
Which then creates globally unique sub-tasks and solves the problem.
Wondering if this is a bug.
The text was updated successfully, but these errors were encountered:
Thanks for the report -- I won't be able to look at this for a bit but if confirmed it is a bug. If you'd like to see this addressed sooner, it would help if you could create a fork/branch with a new demo item showing the behavior -- perhaps two simple long-term sleep tasks that run serially when the pyflow job is given two cores and sufficient memory.
Hello,
I was trying to debug this for a very long time.. It seems that if you have sub-tasks with the same label they will not run in parallel. For example if you have:
Only one "foo" and "bar" will run in parallel, even though their labels are "task-1+foo", "task-1+bar", "task-2+foo", "task-2+bar", "task-3+foo", "task-3+bar"
A quick workaround is this:
Which then creates globally unique sub-tasks and solves the problem.
Wondering if this is a bug.
The text was updated successfully, but these errors were encountered: