feat: use non-presigned url for S3 HEAD requests #971

n1ru4l · 2023-01-09T10:54:36Z

This replaces the AWS s3 SDK for the isomorphic code (Node.js + Cloudflare Workers) with an inlined version of aws4fetch.

The reason for this is that we used to generate pre-signed URLs for HEAD requests instead of doing direct requests with the AWS S3 SDK.
We could not do the requests through the AWS S3 SDK as it provided no way of providing a custom fetch implementation that is able to successfully execute requests on Cloudflare Workers (aws/aws-sdk-js-v3#3104).

There is another issue with the Cloudflare R2 S3 compatible API. We noticed 500 (https://community.cloudflare.com/t/sometimes-r2-response-code-500/430205/14), sometimes 503, and recently also 499 status codes issued for the pre-signed HEAD requests.

Thus, we want to try non-signed HEAD requests instead, to see if that might reduce the number of failures.

Because of mhart/aws4fetch#22 I had to inline the aws4fetch library instead of adding it as a dependency.

changeset-bot · 2023-01-09T10:54:39Z

🦋 Changeset detected

Latest commit: fe1d4ab

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@graphql-hive/client	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions · 2023-01-09T10:57:05Z

🚀 Website Preview

The latest changes to the website are available as preview in: https://4fef2d75.hive-landing-page.pages.dev

github-actions · 2023-01-09T10:57:46Z

🚀 Snapshot Release (`alpha`)

The latest changes of this PR are available as alpha on npm (based on the declared changesets):

Package	Version	Info
`@graphql-hive/cli`	`0.19.11-alpha-20230111132355-d16b423`	npm ↗︎ unpkg ↗︎
`@graphql-hive/client`	`0.22.0-alpha-20230111132355-d16b423`	npm ↗︎ unpkg ↗︎

github-actions · 2023-01-09T11:10:06Z

🐋 This PR was built and pushed to the following Docker images (tag: fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c):

Docker Bake metadata

{
"app": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The app of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/app",
      "build-arg:PORT": "3000",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "app.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The app of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/app",
      "build-arg:PORT": "3000",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "app.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:6f5196912d57226a8f4b51921050dd03813f3b7e78fba0935f14192a2dce48a7",
    "size": 685
  },
  "containerimage.digest": "sha256:6f5196912d57226a8f4b51921050dd03813f3b7e78fba0935f14192a2dce48a7",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/app:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/app:artifacts_api-benchmark"
},
"composition-federation-2": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "Federation 2 Composition Service for GraphQL Hive.",
      "build-arg:IMAGE_TITLE": "graphql-hive/composition-federation-2",
      "build-arg:PORT": "3069",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "Federation 2 Composition Service for GraphQL Hive.",
      "build-arg:IMAGE_TITLE": "graphql-hive/composition-federation-2",
      "build-arg:PORT": "3069",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:7f1448c11f6e4ae43b8f4abcc0f60651843d69c8e3cc493bae091cdeb4f2c459",
    "size": 685
  },
  "containerimage.digest": "sha256:7f1448c11f6e4ae43b8f4abcc0f60651843d69c8e3cc493bae091cdeb4f2c459",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/composition-federation-2:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/composition-federation-2:artifacts_api-benchmark"
},
"docs": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The docs of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/docs",
      "build-arg:PORT": "3000",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "app.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The docs of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/docs",
      "build-arg:PORT": "3000",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "app.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:acfc603fb52cb59505b6bc3bdbb6d0aef5408a7e3cf178ab054b06a6968d3e75",
    "size": 685
  },
  "containerimage.digest": "sha256:acfc603fb52cb59505b6bc3bdbb6d0aef5408a7e3cf178ab054b06a6968d3e75",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/docs:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/docs:artifacts_api-benchmark"
},
"emails": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The emails service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/emails",
      "build-arg:PORT": "3006",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The emails service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/emails",
      "build-arg:PORT": "3006",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:199ff3f0bbd2f08365bba6b4a7d01f477086e9afea3a155d51a03d7bd747911d",
    "size": 685
  },
  "containerimage.digest": "sha256:199ff3f0bbd2f08365bba6b4a7d01f477086e9afea3a155d51a03d7bd747911d",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/emails:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/emails:artifacts_api-benchmark"
},
"rate-limit": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The rate limit service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/rate-limit",
      "build-arg:PORT": "3009",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The rate limit service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/rate-limit",
      "build-arg:PORT": "3009",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:3a05f4a2f15644a221e57d564926cd643825dccf1e83ce5ff4245dd3733fa846",
    "size": 685
  },
  "containerimage.digest": "sha256:3a05f4a2f15644a221e57d564926cd643825dccf1e83ce5ff4245dd3733fa846",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/rate-limit:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/rate-limit:artifacts_api-benchmark"
},
"schema": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The schema service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/rate-limit",
      "build-arg:PORT": "3002",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The schema service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/rate-limit",
      "build-arg:PORT": "3002",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:2341154b2ab3cf8fa761e471dda4863812fc34ea7419bc6027b56242809b8084",
    "size": 685
  },
  "containerimage.digest": "sha256:2341154b2ab3cf8fa761e471dda4863812fc34ea7419bc6027b56242809b8084",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/schema:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/schema:artifacts_api-benchmark"
},
"server": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The server service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/server",
      "build-arg:PORT": "3001",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The server service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/server",
      "build-arg:PORT": "3001",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:837f90b9f9adbff1ed3e1675613aa322681c4c5db5ff0159eeeaad0687c42ca6",
    "size": 685
  },
  "containerimage.digest": "sha256:837f90b9f9adbff1ed3e1675613aa322681c4c5db5ff0159eeeaad0687c42ca6",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/server:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/server:artifacts_api-benchmark"
},
"storage": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The storage service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/storage",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:IMAGE_DESCRIPTION": "The storage service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/storage",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:87a322b9b0d52d44906c0f0372c14a4654c97fde0f1d446ef11b0d72ed049f78",
    "size": 685
  },
  "containerimage.digest": "sha256:87a322b9b0d52d44906c0f0372c14a4654c97fde0f1d446ef11b0d72ed049f78",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/storage:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/storage:artifacts_api-benchmark"
},
"stripe-billing": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The stripe billing service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/stripe-billing",
      "build-arg:PORT": "3010",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The stripe billing service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/stripe-billing",
      "build-arg:PORT": "3010",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:3703ce9d839d21c08ae6442d2d25c01777d42f42b96529ba88070309f2ed07f3",
    "size": 685
  },
  "containerimage.digest": "sha256:3703ce9d839d21c08ae6442d2d25c01777d42f42b96529ba88070309f2ed07f3",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/stripe-billing:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/stripe-billing:artifacts_api-benchmark"
},
"tokens": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The tokens service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/tokens",
      "build-arg:PORT": "3003",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The tokens service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/tokens",
      "build-arg:PORT": "3003",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:7415e27f190e36ac978c75d0651875f918cece29a4309971a39e797e028eabb5",
    "size": 685
  },
  "containerimage.digest": "sha256:7415e27f190e36ac978c75d0651875f918cece29a4309971a39e797e028eabb5",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/tokens:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/tokens:artifacts_api-benchmark"
},
"usage": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage",
      "build-arg:PORT": "3006",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage",
      "build-arg:PORT": "3006",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:26fa9cc4b070b4b04d7a4202c443f84eca9c52e7a453dd4b686193709aa24cc6",
    "size": 685
  },
  "containerimage.digest": "sha256:26fa9cc4b070b4b04d7a4202c443f84eca9c52e7a453dd4b686193709aa24cc6",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/usage:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/usage:artifacts_api-benchmark"
},
"usage-estimator": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage estimator service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage-estimator",
      "build-arg:PORT": "3008",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage estimator service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage-estimator",
      "build-arg:PORT": "3008",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:3b53a4a8c276adda7a8288e1abad6c09e7808818e0ef7e7e0bc8b0a73560bf4d",
    "size": 685
  },
  "containerimage.digest": "sha256:3b53a4a8c276adda7a8288e1abad6c09e7808818e0ef7e7e0bc8b0a73560bf4d",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/usage-estimator:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/usage-estimator:artifacts_api-benchmark"
},
"usage-ingestor": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage-ingestor",
      "build-arg:PORT": "3007",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The usage ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/usage-ingestor",
      "build-arg:PORT": "3007",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:85fe14e860715f25351bf99585520d6c3f478ba27d735bfa03f6007e4f38364e",
    "size": 685
  },
  "containerimage.digest": "sha256:85fe14e860715f25351bf99585520d6c3f478ba27d735bfa03f6007e4f38364e",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/usage-ingestor:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/usage-ingestor:artifacts_api-benchmark"
},
"webhooks": {
  "containerimage.buildinfo/linux/amd64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The webhooks ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/webhooks",
      "build-arg:PORT": "3005",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.buildinfo/linux/arm64": {
    "frontend": "dockerfile.v0",
    "attrs": {
      "add-hosts": "",
      "build-arg:HEALTHCHECK_CMD": "wget --spider -q http://127.0.0.1:${PORT}/_readiness",
      "build-arg:IMAGE_DESCRIPTION": "The webhooks ingestor service of the GraphQL Hive project.",
      "build-arg:IMAGE_TITLE": "graphql-hive/webhooks",
      "build-arg:PORT": "3005",
      "build-arg:RELEASE": "fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c",
      "filename": "services.dockerfile"
    },
    "sources": [
      {
        "type": "docker-image",
        "ref": "docker.io/library/node:18.13.0-slim",
        "pin": "sha256:bc946484118735406562f17c57ddf5fded436e175b6a51f827aa6540ba1e13de"
      }
    ]
  },
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "digest": "sha256:b1237829de6242619f567de16ae6f708e1317f8cd6600431fcb11a5f45013b14",
    "size": 685
  },
  "containerimage.digest": "sha256:b1237829de6242619f567de16ae6f708e1317f8cd6600431fcb11a5f45013b14",
  "image.name": "ghcr.io/kamilkisiela/graphql-hive/webhooks:fe1d4ab0ea2467c5402d85a3b7eaebf71956ef1c,ghcr.io/kamilkisiela/graphql-hive/webhooks:artifacts_api-benchmark"
}
}

n1ru4l · 2023-01-11T10:41:19Z

packages/services/api/src/shared/aws.ts

+  async fetch(input: RequestInfo, init: AwsRequestInit): Promise<Response> {
+    for (let i = 0; i <= this.retries; i++) {
+      const fetched = fetch(await this.sign(input, init));
+      if (i === this.retries) {
+        return fetched; // No need to await if we're returning anyway
+      }
+      const res = await fetched;
+      if (res.status < 500 && res.status !== 429) {
+        return res;
+      }
+      await new Promise(resolve =>
+        setTimeout(resolve, Math.random() * this.initRetryMs * Math.pow(2, i)),
+      );
+    }
+    throw new Error('An unknown error occurred, ensure retries is not negative');
+  }
+}


@kamilkisiela This thing has 5XX retries included

500 retries is a lot (joke)

Maybe we could use https://github.com/elbywan/wretch or https://github.com/developit/redaxios to not do this the manual way? Not saying we should, just wanted to share the libs with you, you decide.

@kamilkisiela I am a fan of being explicit over abstractions so I won't add this myself

why Yoga then? :))

dotansimha

LGTM. Let's test it and see how it behaves :)
@kamilkisiela please review

n1ru4l · 2023-01-11T11:34:21Z

@dotansimha @kamilkisiela I also added a simple 10x retry for the client SDK.

kamilkisiela

Please add a test to check if retry logic works and we're good to merge it

n1ru4l force-pushed the artifacts-api-benchmark branch from d00599c to 0c41506 Compare January 9, 2023 11:02

n1ru4l force-pushed the artifacts-api-benchmark branch 2 times, most recently from 1f3d4a6 to c63257b Compare January 9, 2023 12:13

feat: use aws4fetch fork

eb83c02

n1ru4l force-pushed the artifacts-api-benchmark branch from c63257b to eb83c02 Compare January 9, 2023 12:16

theguild-bot temporarily deployed to staging January 9, 2023 12:24 Inactive

dotansimha requested a review from kamilkisiela January 9, 2023 13:10

n1ru4l added 2 commits January 9, 2023 15:47

weird code

476cb8e

refactor: no need to pass a request

32c7296

n1ru4l commented Jan 11, 2023

View reviewed changes

theguild-bot temporarily deployed to development January 11, 2023 10:46 Inactive

n1ru4l changed the title ~~feat: remove legacy fallback~~ feat: use non-presigned url for S3 HEAD requests Jan 11, 2023

theguild-bot temporarily deployed to staging January 11, 2023 11:05 Inactive

dotansimha approved these changes Jan 11, 2023

View reviewed changes

feat: add retry logic for client SDKs

149a748

kamilkisiela approved these changes Jan 11, 2023

View reviewed changes

n1ru4l added 2 commits January 11, 2023 14:12

fix: refetch implementation + tests

a3ec6e5

fix: thank you cloudflare for throwing 499

fe1d4ab

kamilkisiela merged commit 0abc58b into main Jan 11, 2023

kamilkisiela deleted the artifacts-api-benchmark branch January 11, 2023 14:16

This was referenced Jan 11, 2023

Upcoming Release Changes #816

Merged

Upcoming Release Changes YassinEldeeb/graphql-hive#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: use non-presigned url for S3 HEAD requests #971

feat: use non-presigned url for S3 HEAD requests #971

n1ru4l commented Jan 9, 2023 •

edited

Loading

changeset-bot bot commented Jan 9, 2023 •

edited

Loading

github-actions bot commented Jan 9, 2023 •

edited

Loading

github-actions bot commented Jan 9, 2023 •

edited

Loading

github-actions bot commented Jan 9, 2023 •

edited

Loading

n1ru4l Jan 11, 2023 •

edited

Loading

kamilkisiela Jan 11, 2023

n1ru4l Jan 11, 2023 •

edited

Loading

kamilkisiela Jan 11, 2023

dotansimha left a comment •

edited

Loading

n1ru4l commented Jan 11, 2023

kamilkisiela left a comment

feat: use non-presigned url for S3 HEAD requests #971

feat: use non-presigned url for S3 HEAD requests #971

Conversation

n1ru4l commented Jan 9, 2023 • edited Loading

changeset-bot bot commented Jan 9, 2023 • edited Loading

🦋 Changeset detected

github-actions bot commented Jan 9, 2023 • edited Loading

🚀 Website Preview

github-actions bot commented Jan 9, 2023 • edited Loading

🚀 Snapshot Release (alpha)

github-actions bot commented Jan 9, 2023 • edited Loading

n1ru4l Jan 11, 2023 • edited Loading

Choose a reason for hiding this comment

kamilkisiela Jan 11, 2023

Choose a reason for hiding this comment

n1ru4l Jan 11, 2023 • edited Loading

Choose a reason for hiding this comment

kamilkisiela Jan 11, 2023

Choose a reason for hiding this comment

dotansimha left a comment • edited Loading

Choose a reason for hiding this comment

n1ru4l commented Jan 11, 2023

kamilkisiela left a comment

Choose a reason for hiding this comment

n1ru4l commented Jan 9, 2023 •

edited

Loading

changeset-bot bot commented Jan 9, 2023 •

edited

Loading

github-actions bot commented Jan 9, 2023 •

edited

Loading

github-actions bot commented Jan 9, 2023 •

edited

Loading

🚀 Snapshot Release (`alpha`)

github-actions bot commented Jan 9, 2023 •

edited

Loading

n1ru4l Jan 11, 2023 •

edited

Loading

n1ru4l Jan 11, 2023 •

edited

Loading

dotansimha left a comment •

edited

Loading