DEV Community

Vanshaj Poonia
Vanshaj Poonia

Posted on

Flyte facing issues while running in parallel

While running multiple independent tasks in parallel which in fact is the primary function of map_task
The code that caused the error is as follows:

from flytekit import map_task, task, workflow

@task
def do_something(value: str) -> str:
    print(f"launched: {value}", flush=True)
    time.sleep(60)  # fakes long process time
    return f"{value}-processed"


@workflow
def do_multiple_things() -> list[str]:
    values = ["foo", "bar", "baz"]
    return map_task(do_something)(value=values)
Enter fullscreen mode Exit fullscreen mode

So here is the solution:

Local runs will not do parallel just yet. (Making flyte-kit execute local runs in parallel is part of a broader project that flyte has plans for someday, but no definite timeline).

Top comments (1)

Collapse
 
davosplatz profile image
David Espejo

Hi

Could you add an intro to this post? I'm not sure I follow what a map task is and what this problem is all about?