Add logic to migrate fieldreport number #2349
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sudip-khanal
requested review from
szabozoltan69,
Rup-Narayan-Rajbanshi and
susilnem
susilnem
requested changes
Dec 30, 2024
susilnem
requested changes
Dec 30, 2024
| @@ -43,7 +42,8 @@ def handle(self, *args, **kwargs): | |||
|
|
|||
| # Update the group data if this report has the highest fr number | |||
| if max_fr_num > group_data["highest_fr_num"]: | |||
| self.stdout.write(f"Updating highest fr_num for group (event_id={report.event.id}, country_id={country.id})") | |||
|
|
|||
| print(f"Updating highest fr_num for group (event_id={report.event.id}, country_id={country.id})") | |||
| @@ -53,12 +53,18 @@ def handle(self, *args, **kwargs): | |||
| highest_fr_num = data["highest_fr_num"] | |||
|
|
|||
| if highest_report: | |||
|
|
|||
| print(f"Setting fr_num={highest_fr_num} for report ID={highest_report.id}") | |||
| self.stdout.write("Completed successfully.") | ||
| excluded_reports.update(fr_num=None) | ||
|
|
||
| print(f"Set fr_num=None for reports in group (event_id={event_id}, country_id={country_id})") |
Comment on lines
62
to
63
| excluded_reports = FieldReport.objects.filter(event_id=event_id, countries=country_id).exclude( | ||
| id=highest_report.id |
There was a problem hiding this comment.
Suggested change
| excluded_reports = FieldReport.objects.filter(event_id=event_id, countries=country_id).exclude( | |
| id=highest_report.id | |
| FieldReport.objects.filter(event_id=event_id, countries=country_id).exclude( | |
| id=highest_report.id).update(fr_num=None) |
thenav56
requested changes
Dec 31, 2024
Comment on lines
59
to
62
| # Set fr_num to null for all other reports in the group | ||
| FieldReport.objects.filter(event_id=event_id, countries=country_id).exclude(id=highest_report.id).update( | ||
| fr_num=None | ||
| ) |
There was a problem hiding this comment.
Do we need this? We shouldn't mutate existing data if not required.
| if highest_report: | ||
|
|
||
| highest_report.fr_num = highest_fr_num | ||
| highest_report.save(update_fields=["fr_num"]) |
There was a problem hiding this comment.
Let's use bulk manager
https://github.com/IFRCGo/go-api/blob/develop/main/managers.py#L69
Comment on lines
34
to
40
| if key not in event_country_data: | ||
| event_country_data[key] = { | ||
| "highest_fr_num": 0, | ||
| "report_highest_fr": None, | ||
| } | ||
|
|
||
| group_data = event_country_data[key] |
There was a problem hiding this comment.
Suggested change
| if key not in event_country_data: | |
| event_country_data[key] = { | |
| "highest_fr_num": 0, | |
| "report_highest_fr": None, | |
| } | |
| group_data = event_country_data[key] | |
| group_data = event_country_data[key] = event_country_data.get(key, { | |
| "highest_fr_num": 0, | |
| "report_highest_fr": None, | |
| }) |
| def handle(self, *args, **kwargs): | ||
| suffix_pattern = re.compile(r"#(\d+)") | ||
|
|
||
| reports = FieldReport.objects.filter(summary__icontains="#", event__isnull=False, countries__isnull=False) |
There was a problem hiding this comment.
Let's exclude any event/country field reports with fr_num field with value.
Something like this
# Only process event/country where we don't have fr_num defined
event_country_filter_qs = Report.objects.order_by().values('event', 'countries').annoate(
count=models.Count('id', filter=models.Q(fr_num__isnull=True))
).exclude(count__gte=1)
report_to_process_qs = Report.objects.filter(
# Still slow but better then doing this within python
event__in=event_country_filter_qs.values('event'),
countries__in=event_country_filter_qs.values('countries'),
).distinct()| from api.models import FieldReport | ||
|
|
||
|
|
||
| class Command(BaseCommand): |
There was a problem hiding this comment.
Let's also monitor memory usages... or use iterator/bulk-managers during for loop
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses
Migration of Field Report Data for Dynamic Title Suffixes
Changes
Checklist
Things that should succeed before merging.
Release
If there is a version update, make sure to tag the repository with the latest version.