Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIMS-2002 Properties Export #2641

Merged
merged 6 commits into from
Aug 27, 2024
Merged

PIMS-2002 Properties Export #2641

merged 6 commits into from
Aug 27, 2024

Conversation

dbarkowsky
Copy link
Collaborator

@dbarkowsky dbarkowsky commented Aug 26, 2024

🎯 Summary

PIMS-2002
This is in response to the timeout issue sometimes happening when exporting the properties in a live environment.
Pretty sure the clause that slows things down is this:
where: { Id: In(buildingIds) }
When it is called getting parcels and buildings.
Suspect this is an issue with how TypeORM handles its queries. Almost looks like it's going a circular query at one point.

Changes

  • Broke up the relations and where properties in the TypeORM call into operations in code instead. For some reason this runs faster than letting TypeORM do the joins and filtering. These are the timings I got in local testing:
  // Getting evals, fiscals, and filtering separately: 850-1050ms
  // Getting as as part of joins, where clause: 1587-1672ms
  • If the request for export data fails, it shows a snackbar message and resets the download icon.
  • PID in export is now formatted with padded zeroes and hyphens
  • IsSensitive column added to export

Testing

It's hard to know if this will save time in OpenShift, where the issue seems to be.
Locally, you can test that the download works by exporting the properties table.
Test it doesn't work by stopping the API and hitting the download button again.

🔰 Checklist

  • I have read and agree with the following checklist and am following the guidelines in our Code of Conduct document.
  • I have performed a self-review of my code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation where required.
  • I have tested my changes to the best of my ability.
  • My changes generate no new warnings.

Copy link

🚀 Deployment Information

The Express API Image has been built with the tag: 2641. Please make sure to utilize this specific tag when promoting these changes to the TEST and PROD environments during the API deployment. For more updates please monitor Image Tags Page on Wiki.

Copy link

codeclimate bot commented Aug 26, 2024

Code Climate has analyzed commit ac9cc3e and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 93.6%.

View more on Code Climate.

Copy link

🚀 Deployment Information

The React APP Image has been built with the tag: 2641. Please make sure to utilize this specific tag when promoting these changes to the TEST and PROD environments during the APP deployment. For more updates please monitor Image Tags Page on Wiki.

Copy link
Collaborator

@LawrenceLau2020 LawrenceLau2020 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work well, downloaded all 11736 properties in about 1.5 seconds compared to 7.75 seconds using the TypeORM joins.
Also, the PIDs are nicely formatted with hyphens in the excel, and the sensitive column is also there.

@dbarkowsky dbarkowsky merged commit da4195c into main Aug 27, 2024
13 checks passed
@dbarkowsky dbarkowsky deleted the PIMS-2002-Properties-Export branch August 27, 2024 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants