grid_data api keep refreshing when backfill DAG runs

[yingxuanwangxuan]

Apache Airflow version

2.3.2 (latest released)

What happened

What you think should happen instead

No response

How to reproduce

No response

Operating System

186-Ubuntu

Versions of Apache Airflow Providers

2.3.2

Deployment

Other

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[bhavaniravi]

@yingxuanwangxuan can you add more info

1. When do you see this error?
2. What should happen instead?
3. How can we reproduce this?

Currently, it looks like the grid view is updating because Auto-refresh is enabled

[bbovenzi]

You can manually click the auto-refresh toggle off to prevent this.
Otherwise, those backfilled runs are "pending" and will refresh.

[yingxuanwangxuan]

You can manually click the auto-refresh toggle off to prevent this. Otherwise, those backfilled runs are "pending" and will refresh.
I think this is a problem, can I fix it.

[yingxuanwangxuan]

@yingxuanwangxuan can you add more info

1. When do you see this error?
2. What should happen instead?
3. How can we reproduce this?

Currently, it looks like the grid view is updating because Auto-refresh is enabled

step1 I schedule DAG（airflow_log_cleanup).
step2 I backfill it.
dags backfill airflow_log_cleanup -s 2022-06-11 -e 2022-06-20 --donot-pickle
step3

I wish it didn't refresh automatically all the time.

[bbovenzi]

What solution do you want other than the already existing ability to manually turn off auto-refresh?

[yingxuanwangxuan]

What solution do you want other than the already existing ability to manually turn off auto-refresh?

in [airflow/www/static/js/grid/api/useGridData.ts] by filter manual and scheduled

[yingxuanwangxuan]

What solution do you want other than the already existing ability to manually turn off auto-refresh?

I'm not sure this is a good solution, can I submit a pr to fix it?

[bbovenzi]

Yeah, go for it!

[cloventt]

I'm not sure I understand the reason for this change. Why would you not want the page to auto-refresh? Auto-refresh is useful for seeing the state of running tasks, regardless of whether that task was triggered via a backfill, schedule or manually.

[yingxuanwangxuan]

I'm not sure I understand the reason for this change. Why would you not want the page to auto-refresh? Auto-refresh is useful for seeing the state of running tasks, regardless of whether that task was triggered via a backfill, schedule or manually.

@cloventt
I am using AirFlow1.10. When I tried to upgrade to a higher version. I found this feature auto-refresh is useful for seeing the state of running tasks. When I backfill my task, the state of these tasks is pending which keeps refreshing.

The airflow Grid page maintains backfilled task state by automatically refreshing whether this is the most appropriate way？ Because the backfill command does not re-run completed DAGs within the given period unless we explicitly instruct it to do.

[Gollum999]

I'm a bit confused by this thread. I have been doing a lot of backfilling recently and I personally find the auto-refresh behavior to be quite useful for monitoring progress. It's still not entirely clear to me what the bug is, but disabling auto-refresh for all backfills seems a bit heavy-handed to me.

[cloventt]

I'm in the same camp @Gollum999 , I still can't visualise a situation in which it would be preferred not to automatically refresh the UI when any task or DAG is running/pending, regardless of whether it was started by a backfill.

[potiuk]

I would kindly suggest any of the people who are commenting here, on a closed task to submit a PR fixing it. I think there are a number of ways this can be handled - so if any of you has an idea, constructive PR proposing a fix (especially that there is a linked PR that disabled it that can be used as starting point) is much better approach than commenting on a closed issue.

I heartily invite any of you @cloventt @Gollum999 @yingxuanwangxuan to attempt to do it.

[Gollum999]

@potiuk Well, if it were up to me I would just revert the original PR, but that doesn't seem especially productive. 😄

I was hoping to get some more details from @yingxuanwangxuan (or perhaps @bbovenzi) to better understand the reason behind this change.

Would it help to re-open this thread? Or start a new one?

[bbovenzi]

Looking at this again, maybe we should revert this change. It is possible to turn off auto-refresh when a user doesn't want it. Therefore, the previously functionality is the most flexible for all users. @Gollum999 I'd welcome a PR to change it.

[yingxuanwangxuan]

I am using AirFlow1.10. When I tried to upgrade to a higher version. I found this feature auto-refresh is useful for seeing the state of running tasks. When I backfill my task, the state of these tasks is pending which keeps refreshing. The airflow Grid page maintains backfilled task state by automatically refreshing whether this is the most appropriate way？ Because the backfill command does not re-run completed DAGs within the given period unless we explicitly instruct it to do. | | hefeixuanzi | | ***@***.*** | On 8/25/2022 11:50，David ***@***.***> wrote： I'm not sure I understand the reason for this change. Why would you not want the page to auto-refresh? Auto-refresh is useful for seeing the state of running tasks, regardless of whether that task was triggered via a backfill, schedule or manually. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>