lightning tidb backend failed to restore csv files when trim-last-separator is true

[glorv]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.
   Use lightning tidb backend to restore a csv file.
   tidb-lightning.toml:

[mydumper.csv]
header = false
trim-last-separator = true

[mydumper]
no-schema = true

target table:

create table t (s1 varchar(12), s2 varchar(12), s3 varchar(12));

source file:
test.t.0.csv:

test,test,test,
test,,,

Error log:

[2020/12/23 22:13:33.613 +08:00] [ERROR] [tidb.go:361] ["execute statement failed"] [stmt="REPLACE INTO `test`.`t` VALUES('test','test','test'),('test')"] [rows="[('test','test','test'),('test')]"] [error="Error 1136: Column count doesn't match value count at row 2"]
[2020/12/23 22:13:33.614 +08:00] [ERROR] [restore.go:1804] ["write to data engine failed"] [table=`test`.`t`] [engineNumber=0] [fileIndex=0] [path=test.t.0.csv:0] [task=deliver] [error="Error 1136: Column count doesn't match value count at row 2"]
[2020/12/23 22:13:33.615 +08:00] [ERROR] [restore.go:116] [-] [table=`test`.`t`] [status=written] [error="Error 1136: Column count doesn't match value count at row 2"]
[2020/12/23 22:13:33.615 +08:00] [ERROR] [main.go:91] ["tidb lightning encountered error stack info"] [error="restore table `test`.`t` failed: Error 1136: Column count doesn't match value count at row 2"] [errorVerbose="Error 1136: Column count doesn't match value count at row 2\ngithub.com/pingcap/errors.AddStack\n\t/Users/glorv/go-path/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201029093017-5a7df2af2ac7/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/Users/glorv/go-path/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20201029093017-5a7df2af2ac7/juju_adaptor.go:15\ngithub.com/pingcap/tidb-lightning/lightning/backend.(*tidbBackend).WriteRows\n\t/Users/glorv/go-path/src/github.com/pingcap/tidb-lightning/lightning/backend/tidb.go:367\ngithub.com/pingcap/tidb-lightning/lightning/backend.(*OpenedEngine).WriteRows\n\t/Users/glorv/go-path/src/github.com/pingcap/tidb-lightning/lightning/backend/backend.go:273\ngithub.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).deliverLoop\n\t/Users/glorv/go-path/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:1803\ngithub.com/pingcap/tidb-lightning/lightning/restore.(*chunkRestore).restore.func2\n\t/Users/glorv/go-path/src/github.com/pingcap/tidb-lightning/lightning/restore/restore.go:2007\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1374\nrestore table `test`.`t` failed"]
[2020/12/23 22:13:33.615 +08:00] [ERROR] [main.go:92] ["tidb lightning encountered error"] [error="restore table `test`.`t` failed: Error 1136: Column count doesn't match value count at row 2"]

Root cause analysis:

tidb-lightning/lightning/mydump/csv_parser.go

Lines 532 to 537 in 8510040

	if parser.cfg.TrimLastSep {
	var i int
	for i = len(records); i > 0 && len(records[i-1]) == 0; i-- {
	}
	records = records[:i]
	}

Here, if trim-last-sep is enabled, current logic may trim more than one seps, but tidb backend encoder won't fill the missing fields with default values which cause column count mismatch in the generated statement.