Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce rich progress bars to the SequentialRunner #2209

Open
antonymilne opened this issue Jan 16, 2023 · 2 comments
Open

Introduce rich progress bars to the SequentialRunner #2209

antonymilne opened this issue Jan 16, 2023 · 2 comments
Labels
Issue: Feature Request New feature or improvement to existing feature

Comments

@antonymilne
Copy link
Contributor

antonymilne commented Jan 16, 2023

Description

Add rich progress bars for the SequentialRunner only. Adding it for the ParallelRunner is a separate task.

Context

Originally written by @datajoely in #1464.

  • Today the only way to track runner progress is via the log output
  • The rich.logging.RichHandler allows us to introduce a complimentary set of rich.progress.Progress bars which will render nicely.
  • I've managed to get this sort of working via hooks in my plug-in.
  • However, I've struggled to get these working with ParallelRunner. Multiprocessing is weird since it's hard to share memory.
  • From a plug-in point of view I think it's fine to say that this isn't supported, but if we were doing this natively we could do so directly within the runner or potentially introduce a set of runner hooks that would allow us to manage state better.
  • A really cool end state would be to have a mini-progress bar for each ParallelRunner sub-branch 🚀
@antonymilne antonymilne added the Issue: Feature Request New feature or improvement to existing feature label Jan 16, 2023
@merelcht merelcht changed the title Introduce rich progress bars to the runner Introduce rich progress bars to the SequentialRunner Feb 27, 2023
@merelcht
Copy link
Member

@astrojuanlu Shall we just close all the issues about adding more rich stuff?

@astrojuanlu
Copy link
Member

As much as I'd like fancy progress bars in the Kedro output, I agree we shouldn't do this until we've untangled Kedro's dependency on rich. So maybe we can tackle this in 2 phases: de-rich (things like #3985, #3967 (comment), essentially this milestone) and then re-introduce more interactivity on the CLI at a later stage (Kedro 0.20.x...?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue: Feature Request New feature or improvement to existing feature
Projects
Status: No status
Development

No branches or pull requests

3 participants