Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/psa-metrics #969

Merged
merged 7 commits into from
Feb 23, 2022
Merged

feature/psa-metrics #969

merged 7 commits into from
Feb 23, 2022

Conversation

alexandrastoica
Copy link
Contributor

@alexandrastoica alexandrastoica commented Feb 10, 2022

Adds metrics:

  • upload by type (car, blob, multipart, upload)
  • pin requests count
  • refactors pins statuses listing

BEGIN_COMMIT_OVERRIDE
feat: update metrics and add psa pin requests to metrics
END_COMMIT_OVERRIDE

@alexandrastoica alexandrastoica marked this pull request as draft February 10, 2022 10:25
Copy link
Contributor

@LeslieOA LeslieOA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I know at some point we'll have to consider caching of metrics as proposed on NFT.storage

@alexandrastoica alexandrastoica marked this pull request as ready for review February 11, 2022 10:09
packages/api/src/metrics.js Outdated Show resolved Hide resolved
@@ -325,6 +325,19 @@ BEGIN
END
$$;

CREATE OR REPLACE FUNCTION uploads_by_type(query_type TEXT) RETURNS TEXT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why do we need a procedure here? Shouldn't we just use client count API?
I see that's how it is done for other metrics, but I'm not sure if/why it's needed?

I wonder if was done like this because of the Supabase client issue loading the all table even when just "counting"?
But that can be workarounded by adding range(0, 1)

Probably one for @vasco-santos

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sum we can't do with the client, and that was the main motivation to not use the client. The getPinStatusMetrics using pin_from_status_total I don't recall if there was a reason. Either the cast to pin status, or was just added here with the sums and we didn't consider using the client.

We use the client for other metrics counts, so if we can use it here, I think we should

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexandrastoica did you find a reason to not have it using supabase client?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will change to client! :)


'# HELP web3storage_pins_status_queued_total Total number of pins that are queued.',
'# TYPE web3storage_pins_status_queued_total counter',
`web3storage_pins_status_queued_total ${pinsQueuedTotal}`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to change this. But we need to take into account older values. Should we rename things here to Prometheus best practises and do a sum in Grafana query with the older name?

cc @hugomrdias @alanshaw

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes!

uploads_car_total: 'Car',
uploads_blob_total: 'Blob',
uploads_multipart_total: 'Multipart',
uploads_upload_total: 'Upload'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DB client shouldn't be concerned with the keys we're using in the metrics. An actual upload "type" should be passed to getUploadTypeMetrics.

Copy link
Contributor

@vasco-santos vasco-santos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👌🏼

@alexandrastoica please make sure that when this PR is merged to leave a comment in the Release PR that when the PR is releases we need to update Grafana Dashboard linking to this PR (a Release PR for the API will be created or updated)

Thanks :)

@alexandrastoica alexandrastoica dismissed alanshaw’s stale review February 23, 2022 12:05

Addressed the comment and got LGTM from Vasco

@alexandrastoica alexandrastoica merged commit d647ecb into main Feb 23, 2022
@alexandrastoica alexandrastoica deleted the feature/psa-metrics branch February 23, 2022 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants