perf: Graphql CDN using Stellate #7623

preschian · 2023-10-12T08:41:40Z

Hi @kodadot/internal-dev. Would you like to try Stellate for our GraphQL endpoint? In a simple comparison, it appears to be faster. This could help us if we get a large number of visitors at the same time. Perhaps if the number of visitors is not too high, the effect might not be too noticeable. For now, I’ve set the cache on this endpoint to last for 60 seconds.

Before	After

https://canary.kodadot.xyz/ahk/collection/150	https://deploy-preview-7623--koda-canary.netlify.app/ahk/collection/150

https://stellate.co/docs/graphql-edge-cache/cache-rules

// cache rules

rules: [
  {
    types: [
      'NFTEntity',
      'NFTEntitiesConnection',
      'CollectionEntity',
      'CollectionEntitiesConnection',
      'Event',
      'EventEntity',
      'LastEventEntity'
    ],
    maxAge: 60,
    swr: 600,
    description: 'Cache some types',
  },
],

PR Type

Bugfix
Feature
Refactoring

Context

Related: GraphQL CDN #2743
Related: Perf optimization on landing page #4657

Had issue bounty label?

My DOT address: Payout

Copilot Summary

`🤖 Generated by Copilot at 45af347`

This pull request adds support for the new Stellate indexers, which are a different kind of Subsquid indexer for the NFT gallery. It updates the indexers.ts and constants.ts files to handle both Stellate and Squid indexers and use the appropriate one for each network.

`🤖 Generated by Copilot at 45af347`

SquidEndpoint changes
Stellate indexers join in
Autumn of networks

…ck.stellate.sh' * fix(constants.ts): update stick indexer endpoint to 'https://query-stick.stellate.sh'

netlify · 2023-10-12T08:41:45Z

✅ Deploy Preview for koda-canary ready!

Name	Link
🔨 Latest commit	`e792f52`
🔍 Latest deploy log	https://app.netlify.com/sites/koda-canary/deploys/653f7c8ba1486f0008270c02
😎 Deploy Preview	https://deploy-preview-7623--koda-canary.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

reviewpad · 2023-10-12T08:41:58Z

AI-Generated Summary: This pull request includes changes to two files: indexers.ts and constants.ts. The indexer endpoint for 'ahk' in indexers.ts and 'stick' in constants.ts have been updated to 'https://query-stick.stellate.sh'. The addition of this new URL has also been included in the SquidEndpoint type in indexers.ts.

reviewpad · 2023-10-12T08:42:04Z

Reviewpad Report

⚠️ Warnings

Please link an issue to the pull request

roiLeo

How does it work? does it only cache data?
Will we need it for all indexers? wouldn't we have a problem on the billing?
Do you have acces at some sort of backend dashboard?

preschian · 2023-10-12T09:01:04Z

How does it work? does it only cache data?

It wraps the subsquid endpoint with stellate

before:
user -> https://squid.subsquid.io/stick/graphql

after:
user -> https://query-stick.stellate.sh/ -> cache miss -> https://squid.subsquid.io/stick/graphql
user -> https://query-stick.stellate.sh/ -> cache hit -> return the cache response

Will we need it for all indexers? wouldn't we have a problem on the billing?

for testing maybe we can test with one endpoint first? they have a free plan with unlimited requests. the limitation is only working on 2 endpoints, and need to put their badge

Do you have acces at some sort of backend dashboard?

I just created the account by myself. what we need only subsquid endpoint to use stellate

Jarsen136 · 2023-10-12T09:06:53Z

Caching graphql is a good idea, but I also have some concerns.

I wonder if it supports bypassing the cache. For example, if a user buy a nft we need to fetch the latest information as fast as possible instead of cache.
Is it compatible with the current real-time subscription feature useSubscriptionGraphql? I'm afraid the cache would affect it.

preschian · 2023-10-12T09:12:32Z

Caching graphql is a good idea, but I also have some concerns.

I wonder if it supports bypassing the cache. For example, if a user buy a nft we need to fetch the latest information as fast as possible instead of cache.

we can use custom header to bypass the cache https://stellate.co/docs/graphql-edge-cache/bypass-the-cache

or, not putting all Types in the cache. currently in the rules I put

[
  'NFTEntity',
  'NFTEntitiesConnection',
  'CollectionEntity',
  'CollectionEntitiesConnection',
  'Event',
  'EventEntity',
  'LastEventEntity',
]

Is it compatible with the current real-time subscription feature useSubscriptionGraphql? I'm afraid the cache would affect it.

interesting, do we need wss endpoint for subscription call? seems like Stellate can't support that. ref: https://feedback.stellate.co/feature-requests/p/support-websocket-subscriptions

ws subscription error

unfortunately, we need to adjust the subscription over HTTP instead of WS

yangwao · 2023-10-12T10:11:44Z

Ah GraphCDN => Stellate?

We've been considering this one in the past.

…ss://squid.subsquid.io/stick/graphql' as the WebSocket URL

preschian · 2023-10-12T10:49:41Z

For the subscription query, what if we fallback to the subsquid endpoint? like this c01503c

Ah GraphCDN => Stellate?

We've been considering this one in the past.

yes, cdn for graphql

vikiival · 2023-10-26T09:52:04Z

So looking forward to hear opinion from @kodadot/internal-dev about using stellate.

unfortunately, we need to adjust the subscription over HTTP instead of WS

We can fallback to the sqd endpoint.
the point why we use WebSocket on some parts of the app is the reason that we need fresh data.

But I would be happy to try it on AssetHubs (2 endpoints) and if we would see potential benefit we can scale from there.

roiLeo · 2023-10-26T10:04:36Z

So looking forward to hear opinion from @kodadot/internal-dev about using stellate.

meh.. I have no opinion

But I would be happy to try it on AssetHubs (2 endpoints) and if we would see potential benefit we can scale from there.

yes we could try that first

* fix(TheFooter.vue): add missing import for useTheme * refactor(TheFooter.vue): remove unused import for TranslateResult

sonarcloud · 2023-10-30T09:51:35Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

codeclimate · 2023-10-30T09:51:36Z

Code Climate has analyzed commit e792f52 and detected 0 issues on this pull request.

View more on Code Climate.

preschian · 2023-10-30T09:52:42Z

lets try it then? @vikiival
added Stellate badge in the footer

vikiival · 2023-10-30T11:51:35Z

added Stellate badge in the footer

Shouldn't be the badge in the readme?

yangwao · 2023-11-02T16:33:14Z

Thanks!
pay 50 usd

yangwao · 2023-11-02T16:33:26Z

😍 Perfect, I’ve sent the payout
💵 $50 @ 4.58 USD/DOT ~ 10.917 $DOT
🧗 1xjvRADwdJcnmUCLWerEHR4iGCT5EBTm4D4fzLLg4LcAC9p
🔗 0x3a9f96740a41ae5e790748c4f0b85d17e381cf7562d08bb4aa24e3880fd72967

🪅 Let’s grab another issue and get rewarded!
🪄 github.com/kodadot/nft-gallery/issues

yangwao · 2023-11-02T16:34:19Z

looking for improvement.
300ms cut seems fair, might improve the lighthouse?

preschian · 2023-11-03T07:57:02Z

might improve the lighthouse?

It is probably not significant since I only set the cache to 60 seconds
but if the LCP in some page rely on graphql request, we can test to set a longer cache on that

vikiival · 2023-11-03T08:29:26Z

but if the LCP in some page rely on graphql request,

90% of landing 🥹

preschian · 2023-11-03T09:13:18Z

90% of landing 🥹

aahh, I see. let me check

preschian · 2023-11-03T10:44:19Z

After some checking, our FCP is currently a bit high. Currently, adjustment on LCP will not give significant results. We need to reduce the javascript size on the client

SSR or reducing unused JS in the pages could be helpful in this case, I guess

roiLeo · 2023-11-03T10:46:30Z

SSR or reducing unused JS in the pages could be helpful in this case, I guess

feel free to test on #7725

preschian · 2023-11-03T10:47:20Z

SSR or reducing unused JS in the pages could be helpful in this case, I guess

feel free to test on #7725

checking 👍

preschian · 2023-11-03T10:52:44Z

confirmed, with SSR from #7725 looks better

FCP from 9xx ms -> 6xx ms
Unused JS 3.2 MB -> 2.5 MB

this will help us to increase the scores

* fix(indexers.ts): update ahk indexer endpoint to 'https://query-sti…

45af347

…ck.stellate.sh' * fix(constants.ts): update stick indexer endpoint to 'https://query-stick.stellate.sh'

preschian changed the title ~~pref: stellate graphql~~ perf: stellate graphql Oct 12, 2023

reviewpad bot added the small Pull request is small label Oct 12, 2023

preschian marked this pull request as ready for review October 12, 2023 08:45

preschian requested a review from a team as a code owner October 12, 2023 08:45

preschian requested review from roiLeo and Jarsen136 and removed request for a team October 12, 2023 08:45

reviewpad bot added the waiting-for-review label Oct 12, 2023

preschian marked this pull request as draft October 12, 2023 08:47

preschian added the Open to discussion label Oct 12, 2023

roiLeo reviewed Oct 12, 2023

View reviewed changes

* fix(websocket.ts): add support for INDEXERS.ahk client to return 'w…

c01503c

…ss://squid.subsquid.io/stick/graphql' as the WebSocket URL

preschian changed the title ~~perf: stellate graphql~~ perf: graphql cdn using stellate Oct 12, 2023

preschian changed the title ~~perf: graphql cdn using stellate~~ perf: Graphql CDN using Stellate Oct 12, 2023

preschian added 2 commits October 30, 2023 16:41

Merge branch 'main' into perf/stellate-graphql

5442731

* feat(TheFooter.vue): add Stellate powered by badge to the footer

e792f52

* fix(TheFooter.vue): add missing import for useTheme * refactor(TheFooter.vue): remove unused import for TranslateResult

preschian marked this pull request as ready for review October 30, 2023 09:52

vikiival approved these changes Oct 30, 2023

View reviewed changes

Jarsen136 approved these changes Oct 30, 2023

View reviewed changes

yangwao added the paid pull-request has been paid label Nov 2, 2023

yangwao merged commit 6f7638f into main Nov 2, 2023
13 of 14 checks passed

yangwao deleted the perf/stellate-graphql branch November 2, 2023 16:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: Graphql CDN using Stellate #7623

perf: Graphql CDN using Stellate #7623

preschian commented Oct 12, 2023 •

edited

Loading

netlify bot commented Oct 12, 2023 •

edited

Loading

reviewpad bot commented Oct 12, 2023

reviewpad bot commented Oct 12, 2023

roiLeo left a comment

preschian commented Oct 12, 2023

Jarsen136 commented Oct 12, 2023 •

edited

Loading

preschian commented Oct 12, 2023 •

edited

Loading

yangwao commented Oct 12, 2023

preschian commented Oct 12, 2023

vikiival commented Oct 26, 2023

roiLeo commented Oct 26, 2023

sonarcloud bot commented Oct 30, 2023

codeclimate bot commented Oct 30, 2023

preschian commented Oct 30, 2023

vikiival commented Oct 30, 2023

yangwao commented Nov 2, 2023

yangwao commented Nov 2, 2023

yangwao commented Nov 2, 2023

preschian commented Nov 3, 2023

vikiival commented Nov 3, 2023

preschian commented Nov 3, 2023

preschian commented Nov 3, 2023

roiLeo commented Nov 3, 2023

preschian commented Nov 3, 2023

preschian commented Nov 3, 2023

perf: Graphql CDN using Stellate #7623

perf: Graphql CDN using Stellate #7623

Conversation

preschian commented Oct 12, 2023 • edited Loading

PR Type

Context

Had issue bounty label?

Copilot Summary

🤖 Generated by Copilot at 45af347

🤖 Generated by Copilot at 45af347

netlify bot commented Oct 12, 2023 • edited Loading

✅ Deploy Preview for koda-canary ready!

reviewpad bot commented Oct 12, 2023

reviewpad bot commented Oct 12, 2023

roiLeo left a comment

Choose a reason for hiding this comment

preschian commented Oct 12, 2023

Jarsen136 commented Oct 12, 2023 • edited Loading

preschian commented Oct 12, 2023 • edited Loading

yangwao commented Oct 12, 2023

preschian commented Oct 12, 2023

vikiival commented Oct 26, 2023

roiLeo commented Oct 26, 2023

sonarcloud bot commented Oct 30, 2023

codeclimate bot commented Oct 30, 2023

preschian commented Oct 30, 2023

vikiival commented Oct 30, 2023

yangwao commented Nov 2, 2023

yangwao commented Nov 2, 2023

yangwao commented Nov 2, 2023

preschian commented Nov 3, 2023

vikiival commented Nov 3, 2023

preschian commented Nov 3, 2023

preschian commented Nov 3, 2023

roiLeo commented Nov 3, 2023

preschian commented Nov 3, 2023

preschian commented Nov 3, 2023

preschian commented Oct 12, 2023 •

edited

Loading

`🤖 Generated by Copilot at 45af347`

`🤖 Generated by Copilot at 45af347`

netlify bot commented Oct 12, 2023 •

edited

Loading

Jarsen136 commented Oct 12, 2023 •

edited

Loading

preschian commented Oct 12, 2023 •

edited

Loading