Skip to content
This repository has been archived by the owner on Dec 8, 2021. It is now read-only.

Better progress updates #440

Open
kolbe opened this issue Nov 3, 2020 · 1 comment
Open

Better progress updates #440

kolbe opened this issue Nov 3, 2020 · 1 comment
Labels
feature-request This issue is a feature request

Comments

@kolbe
Copy link

kolbe commented Nov 3, 2020

Feature Request

Is your feature request related to a problem? Please describe:

A large restore today can appear to get "stuck" in some phases, especially "post-processing", where it's possible to see dozens of messages like this over the course of an hour or more:

[2020/11/02 21:53:58.838 +00:00] [INFO] [restore.go:625] [progress] [files="14940/14940 (100.0%)"] [tables="2/3 (66.7%)"] [chunks=14940/14940] [speed(MiB/s)=141.12999972982269] [state=post-processing] []

Describe the feature you'd like:
We should find better ways to communicate incremental progress information for every phrase of restore.

@kolbe kolbe added the feature-request This issue is a feature request label Nov 3, 2020
@glorv
Copy link
Contributor

glorv commented Dec 1, 2020

We are planning to take two steps to optimize the progress log:

  1. Short term optimization. for local backend, add progress for import phase something like [import=66GB/100GB(66%)].
  2. Long term optimization. We will optimize lightning concurrent model to make restore and importer run in pipeline better to avoid the long tail import phase. After this optimization, the current progress will accurately repression the total progress.

@kennytm @Little-Wallace @overvenus PTAL

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature-request This issue is a feature request
Projects
None yet
Development

No branches or pull requests

2 participants