support CastTimeAsDuration in TiFlash #5548

AntiTopQuark · 2022-08-06T10:19:33Z

What problem does this PR solve?

Issue Number: close #5306
related pr:pingcap/tidb#37036

Problem Summary:
support CastTimeAsDuration in TiFlash

What is changed and how it works?

register the cast function
implement the calculation for casting time to duration
add tests of conversion

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Support to pushdown CastTimeAsDuration to TiFlash

1. register the cast function 2. implement the calculation for casting time to duration 3. add unit test of conversion

ti-chi-bot · 2022-08-06T10:19:35Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

SeaRise
ywqzzy

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

sre-bot · 2022-08-06T10:19:41Z

All committers have signed the CLA.

ti-chi-bot · 2022-08-06T10:19:46Z

Welcome @AntiTopQuark!

It looks like this is your first PR to pingcap/tiflash 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to pingcap/tiflash. 😃

AntiTopQuark · 2022-08-06T10:57:29Z

/retest

AntiTopQuark · 2022-08-06T10:58:01Z

/run-all-tests

sre-bot · 2022-08-06T11:21:13Z

Coverage for changed files

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                    347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionsTiDBConversion.h               862               398    53.83%          88                20    77.27%        1971               556    71.79%         494               297    39.88%
Functions/tests/gtest_tidb_conversion.cpp         537               115    78.58%          38                 0   100.00%        1503                 0   100.00%          74                64    13.51%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            1746               648    62.89%         171                27    84.21%        4112               813    80.23%         980               549    43.98%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19020      9231             51.47%    215579  94015        56.39%

full coverage report (for internal network access only)

AntiTopQuark · 2022-08-08T14:10:45Z

/cc @CalvinNeo

AntiTopQuark · 2022-08-08T14:11:25Z

/cc @JaySon-Huang

dbms/src/Functions/FunctionsTiDBConversion.h

1. Split the date and time types 2. Add some comments

dbms/src/Functions/FunctionsTiDBConversion.h

tests/fullstack-test/expr/cast_as_duration.test

dbms/src/Functions/tests/gtest_tidb_conversion.cpp

1. use checkAndGetDataType to get from_type

1. add null values into fullstack-test 2. add const and null test cases into unittest

dbms/src/Functions/FunctionsTiDBConversion.h

AntiTopQuark · 2022-08-11T13:41:21Z

/run-all-tests

sre-bot · 2022-08-11T14:06:35Z

Coverage for changed files

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                    347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionsTiDBConversion.h               857               395    53.91%          87                20    77.01%        1950               557    71.44%         494               295    40.28%
Functions/tests/gtest_tidb_conversion.cpp         558               115    79.39%          38                 0   100.00%        1552                 0   100.00%          74                64    13.51%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            1762               645    63.39%         170                27    84.12%        4140               814    80.34%         980               547    44.18%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19248      9236             52.02%    218945  93946        57.09%

full coverage report (for internal network access only)

AntiTopQuark · 2022-08-12T00:44:41Z

/cc @ywqzzy

dbms/src/Functions/FunctionsTiDBConversion.h

AntiTopQuark · 2022-08-12T13:25:15Z

/run-all-tests

sre-bot · 2022-08-12T13:51:46Z

Coverage for changed files

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                    347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionsTiDBConversion.h               857               393    54.14%          87                20    77.01%        1944               553    71.55%         494               292    40.89%
Functions/tests/gtest_tidb_conversion.cpp         558               115    79.39%          38                 0   100.00%        1552                 0   100.00%          74                64    13.51%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            1762               643    63.51%         170                27    84.12%        4134               810    80.41%         980               544    44.49%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19256      9242             52.00%    219070  94024        57.08%

full coverage report (for internal network access only)

AntiTopQuark · 2022-08-13T09:47:47Z

/cc @ywqzzy

AntiTopQuark · 2022-08-15T00:48:29Z

/cc @Willendless

ywqzzy

LGTM

SeaRise · 2022-08-15T07:28:13Z

/merge

ti-chi-bot · 2022-08-15T07:28:14Z

@SeaRise: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-08-15T07:28:16Z

This pull request has been accepted and is ready to merge.

Commit hash: 003009a

sre-bot · 2022-08-15T07:55:36Z

Coverage for changed files

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                    347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionsTiDBConversion.h               857               395    53.91%          87                20    77.01%        1944               557    71.35%         494               295    40.28%
Functions/tests/gtest_tidb_conversion.cpp         558               115    79.39%          38                 0   100.00%        1552                 0   100.00%          74                64    13.51%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            1762               645    63.39%         170                27    84.12%        4134               814    80.31%         980               547    44.18%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19256      9242             52.00%    219070  93994        57.09%

full coverage report (for internal network access only)

ti-chi-bot · 2022-08-15T08:25:00Z

@AntiTopQuark: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-08-15T08:58:14Z

Coverage for changed files

Filename                                      Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                    347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionsTiDBConversion.h               857               396    53.79%          87                20    77.01%        1944               560    71.19%         494               296    40.08%
Functions/tests/gtest_tidb_conversion.cpp         558               115    79.39%          38                 0   100.00%        1552                 0   100.00%          74                64    13.51%
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                            1762               646    63.34%         170                27    84.12%        4134               817    80.24%         980               548    44.08%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19257      9243             52.00%    219073  94037        57.08%

full coverage report (for internal network access only)

AntiTopQuark added 7 commits August 6, 2022 11:47

fix typo:change \'<\' to \'>\'

a62cfb0

feat: support CastTimeAsDuration in TiFlash

9bf24a3

1. register the cast function 2. implement the calculation for casting time to duration 3. add unit test of conversion

fix typo: add \'tidb_\' prefix

e6765ac

add integration-test for cast time as duration

f4d61aa

fix typo: delete unused '-'

d92f9b9

test: update unittest and integration_test

b964b23

test: update integration_test

e1a17ae

ti-chi-bot added the release-note-none Denotes a PR that doesn't merit a release note. label Aug 6, 2022

ti-chi-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 6, 2022

format: format code

c6f2fbb

Willendless mentioned this pull request Aug 6, 2022

Call For Participation: implement function push down #5092

Open

ti-chi-bot requested a review from CalvinNeo August 8, 2022 14:10

ti-chi-bot requested a review from JaySon-Huang August 8, 2022 14:11

Merge branch 'master' into master

edbac27

ywqzzy reviewed Aug 9, 2022

View reviewed changes

dbms/src/Functions/FunctionsTiDBConversion.h Show resolved Hide resolved

ywqzzy reviewed Aug 9, 2022

View reviewed changes

dbms/src/Functions/FunctionsTiDBConversion.h Outdated Show resolved Hide resolved

ywqzzy self-requested a review August 9, 2022 02:39

feat: support castTimeAsDuration func

085c704

1. Split the date and time types 2. Add some comments

SeaRise reviewed Aug 9, 2022

View reviewed changes

AntiTopQuark added 3 commits August 10, 2022 18:25

update: support CastTimeAsDuration func

ccc41e1

1. use checkAndGetDataType to get from_type

update test: support CastTimeAsDuration func

1850afc

1. add null values into fullstack-test 2. add const and null test cases into unittest

update unittest: format code

bd312d1

Merge branch 'master' into master

977ef42

Willendless reviewed Aug 11, 2022

View reviewed changes

dbms/src/Functions/FunctionsTiDBConversion.h Outdated Show resolved Hide resolved

AntiTopQuark added 2 commits August 11, 2022 15:30

Merge branch 'master' into master

d7aa49b

Merge branch 'master' into master

2a713a1

Merge branch 'master' into master

fcd5834

Willendless reviewed Aug 12, 2022

View reviewed changes

dbms/src/Functions/FunctionsTiDBConversion.h Outdated Show resolved Hide resolved

use ColumnConst to optimize castTimeAsDuration

6757ef8

ti-chi-bot requested a review from Willendless August 15, 2022 00:48

Merge branch 'master' into master

003009a

ywqzzy approved these changes Aug 15, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 15, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 15, 2022

Merge branch 'master' into master

35b37a0

Merge branch 'master' into master

a4cb0ca

ti-chi-bot merged commit 577aada into pingcap:master Aug 15, 2022

ti-chi-bot removed the request for review from Willendless February 8, 2023 01:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support CastTimeAsDuration in TiFlash #5548

support CastTimeAsDuration in TiFlash #5548

AntiTopQuark commented Aug 6, 2022 •

edited

Loading

ti-chi-bot commented Aug 6, 2022 •

edited

Loading

sre-bot commented Aug 6, 2022 •

edited

Loading

ti-chi-bot commented Aug 6, 2022

AntiTopQuark commented Aug 6, 2022

AntiTopQuark commented Aug 6, 2022

sre-bot commented Aug 6, 2022

AntiTopQuark commented Aug 8, 2022

AntiTopQuark commented Aug 8, 2022

AntiTopQuark commented Aug 11, 2022

sre-bot commented Aug 11, 2022

AntiTopQuark commented Aug 12, 2022

AntiTopQuark commented Aug 12, 2022

sre-bot commented Aug 12, 2022

AntiTopQuark commented Aug 13, 2022

AntiTopQuark commented Aug 15, 2022

ywqzzy left a comment

SeaRise commented Aug 15, 2022

ti-chi-bot commented Aug 15, 2022

ti-chi-bot commented Aug 15, 2022

sre-bot commented Aug 15, 2022

ti-chi-bot commented Aug 15, 2022

sre-bot commented Aug 15, 2022

support CastTimeAsDuration in TiFlash #5548

support CastTimeAsDuration in TiFlash #5548

Conversation

AntiTopQuark commented Aug 6, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Aug 6, 2022 • edited Loading

sre-bot commented Aug 6, 2022 • edited Loading

ti-chi-bot commented Aug 6, 2022

AntiTopQuark commented Aug 6, 2022

AntiTopQuark commented Aug 6, 2022

sre-bot commented Aug 6, 2022

Coverage for changed files

Coverage summary

AntiTopQuark commented Aug 8, 2022

AntiTopQuark commented Aug 8, 2022

AntiTopQuark commented Aug 11, 2022

sre-bot commented Aug 11, 2022

Coverage for changed files

Coverage summary

AntiTopQuark commented Aug 12, 2022

AntiTopQuark commented Aug 12, 2022

sre-bot commented Aug 12, 2022

Coverage for changed files

Coverage summary

AntiTopQuark commented Aug 13, 2022

AntiTopQuark commented Aug 15, 2022

ywqzzy left a comment

Choose a reason for hiding this comment

SeaRise commented Aug 15, 2022

ti-chi-bot commented Aug 15, 2022

ti-chi-bot commented Aug 15, 2022

sre-bot commented Aug 15, 2022

Coverage for changed files

Coverage summary

ti-chi-bot commented Aug 15, 2022

sre-bot commented Aug 15, 2022

Coverage for changed files

Coverage summary

AntiTopQuark commented Aug 6, 2022 •

edited

Loading

ti-chi-bot commented Aug 6, 2022 •

edited

Loading

sre-bot commented Aug 6, 2022 •

edited

Loading