Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] [CDC] Loss last batch data in splitSchemaChangeStream #7000

Closed
2 of 3 tasks
dailai opened this issue Jun 17, 2024 · 1 comment · Fixed by #7003
Closed
2 of 3 tasks

[Bug] [CDC] Loss last batch data in splitSchemaChangeStream #7000

dailai opened this issue Jun 17, 2024 · 1 comment · Fixed by #7003
Assignees

Comments

@dailai
Copy link
Contributor

dailai commented Jun 17, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

When a Schema change occurs, the last batch of data after the last Schema change is lost。I use this pr test #6929

SeaTunnel Version

dev

SeaTunnel Config

mysqlcdc_to_mysql_with_schema_change.conf

Running Command

org.apache.seatunnel.connectors.seatunnel.cdc.mysql.MysqlCDCWithSchemaChangeIT#testMysqlCdcWithSchemaEvolutionCase

Error Exception

loss data

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@dailai dailai added the bug label Jun 17, 2024
@dailai
Copy link
Contributor Author

dailai commented Jun 17, 2024

Maybe there's something problem with the code here in org.apache.seatunnel.connectors.cdc.base.source.reader.external.IncrementalSourceStreamFetcher.
截屏2024-06-17 18 08 01

@hailin0 hailin0 self-assigned this Jun 17, 2024
@hailin0 hailin0 added the cdc label Jun 17, 2024
@dailai dailai changed the title [Bug] [Connector-CDC-Base] Loss last batch data in splitSchemaChangeStream [Bug] [Schema Evolution] Loss last batch data in splitSchemaChangeStream Jun 17, 2024
@dailai dailai changed the title [Bug] [Schema Evolution] Loss last batch data in splitSchemaChangeStream [Bug] [CDC] Loss last batch data in splitSchemaChangeStream Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants