You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Given the implementation found at aiida_core/django_jsonb branch, we should take and report the time that is needs for a complete migration of a database of a reasonable size (5m to 10M of nodes).
This is related to issue #2762 and can be inspired by that issue.
Based on the performance we can see if the migration needs improvement.
The text was updated successfully, but these errors were encountered:
I did a fast check on this using a small database of @giovannipizzi (Giovanni can comment on its contents) that had 63K nodes and 1.3M rows in the attributes table (extras almost empty).
The JSONB migration lasted a bit less than 4,5 mins.
For comparison, the migrations from 0008 to 0025 took 17+ mins (with one of them being particularly slower than the others)
If the migration scales linearly with the number of nodes, an 8M nodes database (with similar contents), will take 9 to 10 hours, which seems a reasonable time.
From a quick look, the migrated attributes seem OK (I saw attributes that make sense, numbers remained numbers and not strings etc)
So the migration of the attributes and extras of a 7,3M node database (Davide's) finished in 20 hours.
(django migration: 0034_attributes_extras_settings_json)
More specifically, the timings are the following:
real 1208m12.169s
user 388m30.040s
sys 14m47.921s
And the database contained (rows):
Nodes: 7.351.846
Attributes: 282.505.715
Extras: 3.887.413
Given the implementation found at aiida_core/django_jsonb branch, we should take and report the time that is needs for a complete migration of a database of a reasonable size (5m to 10M of nodes).
This is related to issue #2762 and can be inspired by that issue.
Based on the performance we can see if the migration needs improvement.
The text was updated successfully, but these errors were encountered: