Add invariants validation before writing bad keys to TiKV

[db-will]

Enhancement

Considering the recent issue non-DXF ADD INDEX of a large table (trigger import due to disk quota) writes KV with TS = 0, it is essential to validate invariants before writing invalid keys to TiKV. This is particularly important for processes like Lightning and DDL imports, as these paths directly write to TiKV.

@Benjamin2037

[lance6716]

For the ingest way to write TiKV, we have added a client-side check at

tidb/pkg/lightning/backend/local/region_job.go

Lines 451 to 458 in df0c5d1

	if dataCommitTS == 0 {
	return errors.New("data commitTS is 0")
	}
	req.Chunk = &sst.WriteRequest_Batch{
	Batch: &sst.WriteBatch{
	CommitTs: dataCommitTS,
	},
	}

we will continue to add more invariants checks

[mittalrishabh]

where are you planning to add invariant checks. I think it should be at the lowest layer too : TiKV

[lance6716]

where are you planning to add invariant checks. I think it should be at the lowest layer too : TiKV

Yes, currently we plan to add checks at many different layers:

• the ingest gRPC service at TiKV
• TiDB's ingest client (lightning library)
• data checksum write to / read from lightning's sorting engine
• index scan count
• ...