-
Notifications
You must be signed in to change notification settings - Fork 680
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/remove column schema #477
Feature/remove column schema #477
Conversation
remove using column_schema remove using column_schemas
e91a6dc
to
5f10921
Compare
fix : testcase fix testcase fix: testcase
table init changed
5f10921
to
6aa3872
Compare
Hi Julien! |
Maybe we could keep the old mode and use the new one if the database is compatible even if it's create a lot of additional complexity. I think it will be usefull for people using the library for moving from old MySQL deployment. I agree that the new mode is better. |
@julien-duponchelle @sean-k1 |
Agree a major version increase is a good idea |
I'm not strongly attached to backward compatibility with MySQL 5 even if it will be ideal. If it's too much effort we can make it clear in the documentation that old version require an old version of the library. |
dab68b2
to
10e90a1
Compare
@julien-duponchelle If this value is None, provide logging to the user at the warning level. |
10e90a1
to
0edc051
Compare
Great Let me know when you are ready for a merge |
@julien-duponchelle I'm ready to merge this PR. |
0edc051
to
171b57d
Compare
delete print
cb1c9c0
to
782184a
Compare
d69ba6f
to
1196c06
Compare
@julien-duponchelle |
And voilà |
Resolve #473
Changes
Remove Column Schema Information
Do not rely on current database schemas
Reason
For now, we can benefit from the column schema information, but in the future (since MYSQL 8.0.14) we will rely on optional_meta_data to get table information.
We apologize to those using MYSQL 5, but we no longer support python-mysql-replication prior to version 0.44.0.
We've been relying on the current database to give us values based on incorrect column information if the table at the time of the current event is different from the current database table.
The column information must be obtained from the information in the MYSQL Binlog packets so that we can build a reliable open source project.
Detail
We currently have two problems with relying on the column_schmea of the database.
the table objects in our table_map do not change when DDL Event occurs.
if the table information at the time of the event is different from the column_schema of the current database, we have no way to find out. (because it depends on the current database)
This is a serious problem as it indicates incorrect column information changes on UPDATE, INSERT, and DELETE.
So with this PR, we hope to no longer rely on the database.
In this PR, we don't currently know the column name information because we don't rely on the current database.
Therefore, we set the Table object's column_name_flag = False.
We have now changed all the test case events to work correctly when column_name_flag==True.
After this PR, we are going to write something to change column_name_flag=True when we get column information due to optional_meta_data.
We can also provide it based on the table column information at the time of the event.
(the table objects in our table_map will be changed)
Reference
https://dev.mysql.com/blog-archive/more-metadata-is-written-into-binary-log/