Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] 数据清理二期-订阅实例表 #1896

Closed
CohleRustW opened this issue Oct 27, 2023 · 1 comment
Closed

[FEATURE] 数据清理二期-订阅实例表 #1896

CohleRustW opened this issue Oct 27, 2023 · 1 comment
Assignees
@CohleRustW @neko12583
Labels
backlog 需求初始状态，等待产品进行评估 kind/feature 功能

Comments

@CohleRustW
Copy link
Collaborator

CohleRustW commented Oct 27, 2023 •
edited by ZhuoZhuoCrayon
Loading

背景

为什么需要

功能

你想要什么功能

功能实现

建议的方案

  1. is_latest=true 标识了一个订阅当前所维护的有效实例范围，理论上可以通过清理 is_latest = false 的数据，达到比较好的数据清理效果

企业微信截图_5dca355e-e3e0-4732-b0ba-3d82fc16cf20

  1. 顺带清理 is_latest = false 关联的 SubscriptionTask，及 SubscriptionTask 关联的 PipelineTree

  2. 支持配置清理 xx 天前数据，保留前 xx 天数据，便于排查问题

  3. 策略类型的订阅似乎暂时清理不了，虽然 is_latest=false，但自动巡检任务会同步到 Job，可以评估下清理后页面查看巡检任务的效果，看看UI交互能否接受

  4. 整理出数据清理会影响到的接口，便于测试清理逻辑

实现方案

功能自测

代码变更覆盖功能点需要自测并截图

功能点 1

描述代码变更涉及功能点及自测截图

功能点 2

描述代码变更涉及功能点及自测截图

...
@CohleRustW CohleRustW added kind/feature 功能 backlog 需求初始状态，等待产品进行评估 labels Oct 27, 2023
@ZhuoZhuoCrayon ZhuoZhuoCrayon changed the title [FEATURE] 订阅实例记录表清理 [FEATURE] 数据清理二期-订阅实例表清理 Nov 1, 2023
@ZhuoZhuoCrayon ZhuoZhuoCrayon changed the title [FEATURE] 数据清理二期-订阅实例表清理 [FEATURE] 数据清理二期-订阅实例表 Nov 1, 2023
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 23, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
ac3d5e3
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 23, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
0c0da90
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 23, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
1f7038d
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 30, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
4bb5bda
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 30, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
7ce916c
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 30, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
7c723de
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 30, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
2addf51
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Nov 30, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
c118c18
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 4, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
c478725
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 4, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
2d97689
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 4, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
88c26aa
@neko12583
Copy link
Collaborator

自测截图

清理前数据情况

instance_record表需清理数据量

清理日志

开始清理1

开始清理2

开始清理3

清理后数据情况

清理后数据情况

清理后页面情况以及接口返回

任务历史页面

清理后页面任务历史

一个月前任务展示以及详情

一个月前任务展示

一个月前任务详情

删除数据后task_result接口返回

neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 12, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
22b9fc4
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 19, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
4ee78af
neko12583 added a commit to neko12583/bk-nodeman that referenced this issue Dec 20, 2023
@neko12583
feature: 数据清理二期-订阅实例表(closed TencentBlueKing#1896)
506229a
chalice-1831 added a commit to chalice-1831/bk-nodeman that referenced this issue Apr 26, 2024
@chalice-1831
feat: 支持周期清理订阅实例记录(closed TencentBlueKing#1896)
24ff795
chalice-1831 added a commit to chalice-1831/bk-nodeman that referenced this issue Apr 26, 2024
@chalice-1831
feat: 支持周期清理订阅实例记录(closed TencentBlueKing#1896)
eec254c
chalice-1831 added a commit to chalice-1831/bk-nodeman that referenced this issue Jun 5, 2024
@chalice-1831
feat: 支持周期清理订阅实例记录(closed TencentBlueKing#1896)
a61263b
chalice-1831 added a commit to chalice-1831/bk-nodeman that referenced this issue Jun 5, 2024
@chalice-1831
feat: 支持周期清理订阅实例记录(closed TencentBlueKing#1896)
cbe4ff2
ZhuoZhuoCrayon pushed a commit that referenced this issue Jun 10, 2024
@chalice-1831 @ZhuoZhuoCrayon
feat: 支持周期清理订阅实例记录(closed #1896)
daadcc6
ZhuoZhuoCrayon added a commit to ZhuoZhuoCrayon/bk-nodeman that referenced this issue Jun 11, 2024
@ZhuoZhuoCrayon
feat: 提供任务执行数据定期清理的能力 (closed TencentBlueKing#1896)
2e80a28
ZhuoZhuoCrayon added a commit that referenced this issue Jun 11, 2024
@ZhuoZhuoCrayon
feat: 提供任务执行数据定期清理的能力 (closed #1896)
787043d
ZhuoZhuoCrayon added a commit that referenced this issue Jun 11, 2024
@ZhuoZhuoCrayon
minor: 指针偏移，删除去重 (fixed #1896)
cbad14f
ZhuoZhuoCrayon added a commit to ZhuoZhuoCrayon/bk-nodeman that referenced this issue Jun 11, 2024
@ZhuoZhuoCrayon
minor: 指针偏移，删除去重 (fixed TencentBlueKing#1896)
bbfce28
@wyyalt wyyalt closed this as completed in 5858c7d Jun 13, 2024
wyyalt pushed a commit that referenced this issue Jun 13, 2024
@ZhuoZhuoCrayon @wyyalt
feat: 提供任务执行数据定期清理的能力 (closed #1896)
11c21b0
wyyalt pushed a commit that referenced this issue Jun 13, 2024
@ZhuoZhuoCrayon @wyyalt
minor: 指针偏移，删除去重 (fixed #1896)
067c39a
wyyalt pushed a commit that referenced this issue Jun 17, 2024
@chalice-1831 @wyyalt
feat: 支持周期清理订阅实例记录(closed #1896)
2a70ae6
wyyalt pushed a commit that referenced this issue Jun 17, 2024
@ZhuoZhuoCrayon @wyyalt
feat: 提供任务执行数据定期清理的能力 (closed #1896)
389d726
wyyalt pushed a commit that referenced this issue Jun 17, 2024
@ZhuoZhuoCrayon @wyyalt
minor: 指针偏移，删除去重 (fixed #1896)
dd8dc9a
chalice-1831 added a commit to chalice-1831/bk-nodeman that referenced this issue Jul 18, 2024
@chalice-1831
feat: 支持周期清理订阅实例记录(closed TencentBlueKing#1896)
190e451
chalice-1831 pushed a commit to chalice-1831/bk-nodeman that referenced this issue Jul 18, 2024
@ZhuoZhuoCrayon @chalice-1831
feat: 提供任务执行数据定期清理的能力 (closed TencentBlueKing#1896)
cc44a85
chalice-1831 pushed a commit to chalice-1831/bk-nodeman that referenced this issue Jul 18, 2024
@ZhuoZhuoCrayon @chalice-1831
minor: 指针偏移，删除去重 (fixed TencentBlueKing#1896)
13eae8d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CohleRustW CohleRustW

@neko12583 neko12583

Labels
backlog 需求初始状态，等待产品进行评估 kind/feature 功能
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@CohleRustW @neko12583