Skip to content
This repository has been archived by the owner on Aug 1, 2024. It is now read-only.

chore: update dbcopyall8 command #1204

Merged
merged 4 commits into from
Oct 11, 2023
Merged

Conversation

mumarkhan999
Copy link
Contributor

@mumarkhan999 mumarkhan999 commented Oct 9, 2023

The previous dbcoopyall8 command causing operational errors

The occurrence of the operational errors: table already exists issue can be traced to the following:

  • During the provisioning process, the mysql80 container is populated with the most recent data, (new migrations/tables).
  • When data is transferred from mysql57 using dbcopyall8 command, the newly created tables persist in the mysql80 container. But, the django_migrations table becomes outdated in this process.
  • Consequently, the records for any new migration are removed from the django_migrations table. This leads to a failure when attempting to execute dev.migrate.

This PR is to fix the above-mentioned issue.
It will wipe out all the data of mysql8 before actually dumping mysql57 data inside mysql80 container.

@mumarkhan999 mumarkhan999 force-pushed the umar/update-dbcopyall8-command branch 5 times, most recently from 7f25326 to caabc27 Compare October 9, 2023 12:44
Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
@mumarkhan999 mumarkhan999 force-pushed the umar/update-dbcopyall8-command branch 3 times, most recently from 7a6af20 to bfa59f7 Compare October 10, 2023 16:18
Makefile Outdated Show resolved Hide resolved
docs/manual_upgrades.rst Outdated Show resolved Hide resolved
$(MAKE) dev.wait-for.mysql57+mysql80
docker compose exec -T mysql80 mysql -uroot mysql < provision-mysql80.sql
$(MAKE) $(_db_copy8_targets)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The copying is now failing for me, with the following error:

docker compose exec mysql57 mysqldump "credentials" > .dev/credentials.sql
docker compose exec -T mysql80 mysql "credentials" < .dev/credentials.sql
ERROR 2013 (HY000) at line 621: Lost connection to MySQL server during query
make[1]: *** [Makefile:481: dev.dbcopy8.credentials] Error 1

If I dump the logs just before this statement runs, I get a line about "Temporary server started." This temporary server is started and then stopped before the regular mysql server runs. I don't see this in the mysql57 logs. This might mean that mysql80 needs a different availability check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I move the sleep 10 to below the wait-for line, it works. Something is still very strange here, because I can run various checks against MySQL immediately between the provision-mysql80.sql and _db_copy8_targets lines and they always pass, even when run repeatedly in short succession.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think I've got it! The problem is that the mysql command is talking to the temporary server over a UNIX socket. The solution seems to be to tell mysql to use TCP. I'll submit a separate PR for that, though.

@mumarkhan999 mumarkhan999 force-pushed the umar/update-dbcopyall8-command branch from 285c4b6 to f3c72cd Compare October 11, 2023 14:21
@mumarkhan999
Copy link
Contributor Author

Closing this to re-run openedx/cla. It's stuck for an unknown reason.

@mumarkhan999 mumarkhan999 reopened this Oct 11, 2023
@mumarkhan999 mumarkhan999 merged commit e0f04a7 into master Oct 11, 2023
22 checks passed
@mumarkhan999 mumarkhan999 deleted the umar/update-dbcopyall8-command branch October 11, 2023 14:41
nsprenkle pushed a commit that referenced this pull request Nov 21, 2023
* chore: update dbcopyall8 command
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants