-
Notifications
You must be signed in to change notification settings - Fork 167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add stats routes #1400
feat: add stats routes #1400
Conversation
0bb2f3d
to
ae55d8e
Compare
Deploying with Cloudflare Pages
|
This implements the materialized view implementation i discussed with @dchoi27. This should be good enough for 10's of millions of records. Locally, on a postgres instance with 15 million records, it takes roughly 10-13 seconds, sometimes closer to 6. This is much easier to refresh since it is all postgres side, but it is not the fastest solution. This solution is exact though, so numbers are trustworthy. As noted in my conversation with @dchoi27, the more robust solution should this one be rendered too slow or not robust enough, would be to cash the previous values and programmatically calculate the flow values from the 2 diffed values (previous count and count in past 7 days, added together as current total count). |
Fwiw on a timeseries database i have, this query takes 1.8s on 10 million records, so this is indicative that indexing plays a role in this query speed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM, just a minor request to follow the existing naming convention for the metric names.
Also, in main we're now timing the queries so we can debug more easily as the data size grows - can you please rebase/merge with main and add the withTimeLog
function call around your queries? 🙏
…ishes a refresh pattern
fcaca9b
to
cabf531
Compare
Working on stats routes, these are currently written while not having a local setup to test against.
I'll be testing over the next few days.