Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disttask: record subtask start/update time #46361

Merged
merged 4 commits into from
Aug 24, 2023
Merged

Conversation

D3Hunter
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #45990

Problem Summary:

What is changed and how it works?

  • record subtask start/update time
  • extract StartSubtask, and some refactor

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 23, 2023
@D3Hunter
Copy link
Contributor Author

/check-issue-triage-complete

@D3Hunter
Copy link
Contributor Author

/ok-to-test

@ti-chi-bot ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Aug 23, 2023
@D3Hunter D3Hunter changed the title dist framework: record subtask start/update time disttask: record subtask start/update time Aug 23, 2023
@codecov
Copy link

codecov bot commented Aug 23, 2023

Codecov Report

Merging #46361 (2751f7e) into master (1ea6499) will decrease coverage by 0.6443%.
Report is 20 commits behind head on master.
The diff coverage is 81.8181%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #46361        +/-   ##
================================================
- Coverage   73.3647%   72.7205%   -0.6443%     
================================================
  Files          1288       1312        +24     
  Lines        393441     400671      +7230     
================================================
+ Hits         288647     291370      +2723     
- Misses        86427      90862      +4435     
- Partials      18367      18439        +72     
Flag Coverage Δ
integration 25.6622% <0.0000%> (?)
unit 73.3970% <100.0000%> (+0.0322%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 54.0444% <ø> (ø)
parser 85.0706% <ø> (+0.0001%) ⬆️
br 47.8484% <ø> (-4.3928%) ⬇️

@D3Hunter D3Hunter requested a review from ywqzzy August 23, 2023 10:27
@D3Hunter
Copy link
Contributor Author

/retest

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 24, 2023

@ywqzzy: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

StartTime time.Time
// UpdateTime is the time when the subtask is updated.
// it can be used as subtask end time if the subtask is finished.
// it's 0 if it hasn't started yet.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since updteTime is end time, when it is 0, then not means not start.

Copy link
Contributor Author

@D3Hunter D3Hunter Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since updteTime is end time

it's not the end time, it can be used as subtask end time ONLY IF the subtask is finished.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what I mean is “it's 0 if it hasn't started yet.” is not correct,will startsubtask update both starttime and updatetime during starttime?

@@ -73,6 +73,7 @@ type Manager struct {
// cancelFunc is used to fast cancel the scheduler.Run.
handlingTasks map[int64]context.CancelFunc
}
// server id, it's host:port for now.
Copy link
Collaborator

@Benjamin2037 Benjamin2037 Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// server id, it's host:port for now.
// id, it's host:port for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the manager's id == scheduler's id == server id

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

server id is will cause confuse, since the name of struct has explain what id for, let's delete server id.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it means it same as server id, updated

cancel context.CancelFunc
ctx context.Context
cancel context.CancelFunc
// server id, it's host:port for now.
Copy link
Collaborator

@Benjamin2037 Benjamin2037 Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// server id, it's host:port for now.
// id, it's host:port for now.

@D3Hunter
Copy link
Contributor Author

/test tiprow_fast_test

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 24, 2023

@D3Hunter: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test build
  • /test canary-scan-security
  • /test check-dev
  • /test check-dev2
  • /test mysql-test
  • /test pingcap/tidb/canary_ghpr_unit_test
  • /test pull-br-integration-test
  • /test pull-common-test
  • /test pull-e2e-test
  • /test pull-integration-common-test
  • /test pull-integration-copr-test
  • /test pull-integration-ddl-test
  • /test pull-integration-jdbc-test
  • /test pull-integration-mysql-test
  • /test pull-mysql-connector-test
  • /test pull-sqllogic-test
  • /test pull-tiflash-test
  • /test unit-test

The following commands are available to trigger optional jobs:

  • /test pull-notify-when-compatibility-sections-changed

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tidb/ghpr_build
  • pingcap/tidb/ghpr_check
  • pingcap/tidb/ghpr_check2
  • pingcap/tidb/ghpr_mysql_test
  • pingcap/tidb/ghpr_unit_test

In response to this:

/test tiprow_fast_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ywqzzy
Copy link
Contributor

ywqzzy commented Aug 24, 2023

/LGTM

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Aug 24, 2023
Copy link
Collaborator

@Benjamin2037 Benjamin2037 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Aug 24, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 24, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-08-24 02:50:15.319872435 +0000 UTC m=+1376979.868888419: ☑️ agreed by ywqzzy.
  • 2023-08-24 04:31:06.316240629 +0000 UTC m=+1383030.865256601: ☑️ agreed by Benjamin2037.

Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link

ti-chi-bot bot commented Aug 24, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Benjamin2037, GMHDBJD, ywqzzy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Aug 24, 2023
@ti-chi-bot ti-chi-bot bot merged commit c38f0c1 into master Aug 24, 2023
6 of 9 checks passed
@ti-chi-bot ti-chi-bot bot deleted the subtask-startend-time branch August 24, 2023 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

record start/end time for subtask
4 participants