Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: track content length histogram #1206

Merged
merged 2 commits into from
Feb 3, 2022

Conversation

vasco-santos
Copy link
Contributor

@vasco-santos vasco-santos commented Feb 1, 2022

Adds content length histogram tracking, as well as content length of cached responses, so that we can compare the amount of data we had cached

Closes #1181

@vasco-santos vasco-santos force-pushed the feat/track-content-length-histogram branch from 8d6c61c to 30ccb60 Compare February 1, 2022 10:30
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Feb 1, 2022

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 9ed90c3
Status: ✅  Deploy successful!
Preview URL: https://47f39b7e.nft-storage-1at.pages.dev

View logs

@vasco-santos vasco-santos force-pushed the feat/track-content-length-histogram branch from 30ccb60 to fffca73 Compare February 1, 2022 10:47
@vasco-santos vasco-santos requested a review from alanshaw February 1, 2022 11:02
@vasco-santos vasco-santos force-pushed the feat/track-content-length-histogram branch 2 times, most recently from fc55049 to e6ce43d Compare February 1, 2022 16:14
@vasco-santos vasco-santos force-pushed the feat/track-content-length-histogram branch from e6ce43d to fb5065f Compare February 3, 2022 10:29
Comment on lines 50 to 55
this.totalContentLengthBytes =
(await this.state.storage.get(TOTAL_CONTENT_LENGTH_BYTES_ID)) || 0
// Total cached content length responses
this.totalCachedContentLengthBytes =
(await this.state.storage.get(TOTAL_CACHED_CONTENT_LENGTH_BYTES_ID)) ||
0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These need to be BigInt. Apparently DO storage supports every type supported by the Structured Clone algorithm which includes BigInt thankfully.

https://developers.cloudflare.com/workers/runtime-apis/durable-objects#transactional-storage-api


// We will count occurences per bucket where content size is less or equal than bucket value
export const contentLengthHistogram = [
0.5, 1, 2, 5, 25, 50, 100, 500, 1000, 5000, 10000, 15000, 20000, 30000, 32000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these need to be converted to bytes...?

@@ -70,6 +77,26 @@ test('Gets Metrics content', async (t) => {
const response = await mf.dispatchFetch('http://localhost:8787/metrics')
const metricsResponse = await response.text()

// content length of responses is 23
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

? is zero below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean below? Want to get the metrics before the requests and see if content length is zero?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added that case

@vasco-santos vasco-santos requested a review from alanshaw February 3, 2022 13:53
@vasco-santos vasco-santos merged commit 8723412 into main Feb 3, 2022
@vasco-santos vasco-santos deleted the feat/track-content-length-histogram branch February 3, 2022 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Limit gateway cache size and keep track of file size distribution
2 participants