-
Notifications
You must be signed in to change notification settings - Fork 9
Old page data migration
Recommended practices and scripts for old page data migrations.
It highly depends on the original state of your articles, however, manual migration is a great opportunity to revise and update them.
Automatically copy all the old users, without any login info and store the id mapping. Let users create their new credentials and then merge them (using merge tool). Don't forget to update id mappings accordingly.
4 thinks are (will be) merged. Kaspar (old ksp), fks (old fks), kms (old kms), random csv filed provided by Mato Badin.
There is a script https://github.com/trojsten/web/blob/master/trojsten/people/management/commands/migrate_ksp_kaspar.py that can be run on a web-server and that will connect to kaspar.
To run locally (for testing), you need to ask someone to dump kaspar for you (there is a column with fancy privileges). Then restore it locally and change corresponding variables in settings/common.py. It is a postgres database, so there is not much of a trouble. However you need to create a few new users to run smoothly (mio, kaspar, kaspar_ro, brano ... and so on.). -O Switch does not help, because life would be too easy that way.
You can download kaspar dump from my (vlejd) home directory on element /home/vlejd/data/
Time for MySql. Some database architecture decision are questionable, but nothing terribly painful to do. No problems with restoring.
Migration script needs csv dumps of a few tables.
You can either ask someone to give you the mysql dump and run commands described in trojsten/people/management/commands/migrate_kms_csv.py, or you can download them from my (vlejd) home directory on element /home/vlejd/data/kms
We are no able (to lazy) to properly parse people addresses.
Run https://github.com/trojsten/web/blob/master/trojsten/people/management/commanda/migrate_kms_csv.py
1.1G MySql file. No problems with restoring.
Migration script needs csv dumps of a few tables.
Very similar to KMS. You can either ask someone to give you the mysql dump and run commands described in trojsten/people/management/commands/migrate_fks_csv.py (note than one nice join over there), or you can download them from my (vlejd) home directory on element /home/vlejd/data/fks
Run https://github.com/trojsten/web/blob/master/trojsten/people/management/commanda/migrate_fks_csv.py
We are also migrating some random csv files from events like 30 rokov fks.
They can be found in my (vlejd) home directory on element /home/vlejd/data/
Run https://github.com/trojsten/web/blob/master/trojsten/people/management/commanda/migrate_30rokovfks1_csv.py and https://github.com/trojsten/web/blob/master/trojsten/people/management/commanda/migrate_30rokovfks2_csv.py. Note that each one of these is for one file (those have two formats).
WARNING. Those scripts will not write to database by default! They run on "dry mode" by default.
Scripts described above have two common switches. --fast will only migrate small portion of user. You can check if everything is set up correctly and --wet_run will actually write something to database.
TODO: There will be a script, that converts filesystem structure with pdf statements and solutions, txt metadata (deadlines, task names, task points) and csv results to the correct database instances.
## Events
If you had event attendees in the old database migrate them automatically using user mappings. Otherwise, create old important events manually (using action wizard (TODO: not implemented yet)). Event has it's own wiki article subtree, so you can migrate any content.
Optionally, it is possible to migrate also old user submits. This may, however, be a complicated process, that requires deep knowledge of both old and new submit systems. Try at your own risk. Not done yet.