Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: fix The JSON Data can not import to TiDB correctly by load data (#21043) #21074

Merged
merged 7 commits into from
Nov 25, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #21043 to release-4.0


What problem does this PR solve?

Issue Number: close #20136

Problem Summary:

What is changed and how it works?

What's Changed:

change load data reader

How it Works:

load data will consistent with mysql

Related changes

Check List

Tests

  • Manual test (add detailed scripts or steps below)
> cat load_json.csv
"aaa"
"{""itemRangeType"":0,""itemContainType"":0,""shopRangeType"":0}"
"{""itemRangeType"":0,""itemContainType"":0,""shopRangeType"":1,""shopJson"":""[{\""id\"":\""A1234\"",\""shopName\"":\""AAAAAA\""}]""}"
"{""itemRangeType"":0,""itemContainType"":0,""shopRangeType"":1,""shopJson"":""[{\""id\"":\""A1234\"",\""shopName\"":\""AAAAAA\""}]""}"

mysql> drop table if exists load_json;
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> CREATE TABLE `load_json` ( `aaa` VARCHAR ( 2000 ) );
Query OK, 0 rows affected (0.00 sec)

mysql> load data local infile '/data/load_json.csv' ignore  into table load_json columns terminated by ',' enclosed by '"' escaped by '' ignore 1 lines (`aaa`); 
Query OK, 3 rows affected (0.01 sec)
Records: 3  Deleted: 0  Skipped: 0  Warnings: 0

mysql> select * from load_json; 
+---------------------------------------------------------------------------------------------------------------------+
| aaa                                                                                                                 |
+---------------------------------------------------------------------------------------------------------------------+
| {"itemRangeType":0,"itemContainType":0,"shopRangeType":0}                                                           |
| {"itemRangeType":0,"itemContainType":0,"shopRangeType":1,"shopJson":"[{\"id\":\"A1234\",\"shopName\":\"AAAAAA\"}]"} |
| {"itemRangeType":0,"itemContainType":0,"shopRangeType":1,"shopJson":"[{\"id\":\"A1234\",\"shopName\":\"AAAAAA\"}]"} |
+---------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

Side effects

  • Performance regression
    • Consumes more CPU
      No
    • Consumes more MEM
      No
  • Breaking backward compatibility
    No

Release note

  • fix The JSON Data can not import to TiDB correctly by load data

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot you're already a collaborator in bot's repo.

Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 16, 2020
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 17, 2020
@SunRunAway
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 20609
  • 21073
  • 21073
  • 21019
  • 21055
  • 20413

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 17, 2020
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@ichn-hu
Copy link
Contributor

ichn-hu commented Nov 24, 2020

/run-all-tests

@ichn-hu
Copy link
Contributor

ichn-hu commented Nov 24, 2020

/run-all-tests

@SunRunAway
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 21249
  • 21103

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit 6926836 into pingcap:release-4.0 Nov 25, 2020
@SunRunAway SunRunAway deleted the release-4.0-31ddba99da38 branch November 25, 2020 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants