tidb-lightning failed to parsed sql exported by mydumper.

[jianhaiqing]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do?

DDL:

/*!40101 SET NAMES binary*/;
/*!40014 SET FOREIGN_KEY_CHECKS=0*/;
CREATE TABLE  exam_detail (
  resourceid varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
   level_score_statistic json DEFAULT NULL,
  PRIMARY KEY (resourceid),
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

sql:

/*!40101 SET NAMES binary*/;
/*!40014 SET FOREIGN_KEY_CHECKS=0*/;
INSERT INTO exam_detail VALUES
("00004aedb3584bxxxx4254e482dd92d2",CONVERT("{}" USING UTF8MB4)),
("0000b8930bb24eyyyy2ce4ccb29dd91e",CONVERT("{\"52\": 1, \"54\": 1, \"68\": 1, \"71\": 1, \"72\": 2, \"75\": 1, \"77\": 1, \"78\": 1, \"79\": 1, \"82\": 1, \"85\": 1, \"86\": 2, \"89\": 3, \"91\": 1, \"93\": 2, \"95\": 1, \"98\": 2, \"100\": 3, \"101\": 2, \"102\": 1, \"103\": 1, \"104\": 3, \"105\": 1, \"106\": 1, \"107\": 1, \"108\": 2, \"109\": 2, \"110\": 3, \"111\": 1, \"112\": 2, \"113\": 1, \"114\": 1, \"115\": 1, \"117\": 1, \"126\": 1}" USING UTF8MB4));

2. What did you expect to see?

tidb-lightning can load these sql into tidb successfully.

3. What did you see instead?

2018/12/16 03:47:17.054 main.go:52: [error] tidb lightning encountered error:Syntax error
github.com/pingcap/tidb-lightning/lightning/mydump.(*ChunkParser).lex
        /home/jenkins/workspace/build_tidb_lightning_2.1/go/src/github.com/pingcap/tidb-lightning/lightning/mydump/parser_generated.go:1283
github.com/pingcap/tidb-lightning/lightning/mydump.(*ChunkParser).ReadRow
        /home/jenkins/workspace/build_tidb_lightning_2.1/go/src/github.com/pingcap/tidb-lightning/lightning/mydump/parser.go:128
github.com/pingcap/tidb-lightning/lightning/mydump.(*ChunkParser).ReadChunks
        /home/jenkins/workspace/build_tidb_lightning_2.1/go/src/github.com/pingcap/tidb-lightning/lightning/mydump/parser.go:178
github.com/pingcap/tidb-lightning/lightning/mydump.splitExactChunks
        /home/jenkins/workspace/build_tidb_lightning_2.1/go/src/github.com/pingcap/tidb-lightning/lightning/mydump/region.go:145
github.com/pingcap/tidb-lightning/lightning/mydump.(*RegionFounder).MakeTableRegions.func1
        /home/jenkins/workspace/build_tidb_lightning_2.1/go/src/github.com/pingcap/tidb-lightning/lightning/mydump/region.go:100
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1333
/data/tidb/mydumper/exam_detail.sql

4. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?

v2.1.1

5. which tool are you using?

./bin/mydumper  -V
mydumper 0.9.5, built against MySQL 5.7.23

6. what versionof tool are you using (pump -V or tidb-lightning -V or syncer -V)?

./bin/tidb-lightning -V
Release Version: v2.1.0-4-g510f71e
Git Commit Hash: 510f71e1fc653e2baf9b8539c02010e16a4f8977
Git Branch: release-2.1
UTC Build Time: 2018-12-12 05:15:00
Go Version: go version go1.11.2 linux/amd64

[kennytm]

Thanks for the report, this will be fixed in TiDB Lighting v2.1.2.