Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gateways hangs or returns an error when using multiple shareable levels #3116

Open
nogates opened this issue Aug 14, 2024 · 0 comments
Open

Gateways hangs or returns an error when using multiple shareable levels #3116

nogates opened this issue Aug 14, 2024 · 0 comments

Comments

@nogates
Copy link
Contributor

nogates commented Aug 14, 2024

Issue Description

hello
We are in the process of migrating fields and types between subgraphs. For this reason, we have many levels of shareble items across the two subrgaphs. Consider this examplen of product-legacy subgraph


extend schema
  @link(
    url: "https://specs.apollo.dev/federation/v2.8"
    import: [
      "@key"
      "@shareable"
      "@provides"
      "@external"
      "@tag"
      "@extends"
      "@override"
      "@inaccessible"
      "@composeDirective"
    ]
  )

interface Product @key(fields: "id", resolvable: true) {
  id: ID!
  description: String
  metadata: [ProductMetadataValue!]!
}

type Computer implements Product @key(fields: "id") {
  id: ID!
  description: String
  metadata: [ProductMetadataValue!]! @shareable
  cpu: String
}

type Monitor implements Product @key(fields: "id") {
  id: ID!
  description: String
  metadata: [ProductMetadataValue!]! @shareable
  resolution: String
}

type Mouse implements Product @key(fields: "id") {
  id: ID!
  description: String
  metadata: [ProductMetadataValue!]! @shareable
  dpi: String
}

interface ProductMetadataValue
  @key(fields: "_fieldId _productId", resolvable: true) {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  productMetadata: ProductMetadata!
}

interface ProductMetadata {
  id: ID!
  createdAt: String!
  createdBy: User
}

type TextProductValue implements ProductMetadataValue
  @key(fields: "_fieldId _productId")
  @shareable {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  textValue: String!
  productMetadata: TextProductMetadata!
}

type TextProductMetadata implements ProductMetadata @shareable {
  id: ID!
  createdAt: String!
  createdBy: User
}

interface User @key(fields: "id", resolvable: true) {
  email: String
  id: ID!
}

type Agent implements User @key(fields: "id", resolvable: true) @shareable {
  email: String
  id: ID!
  name: String
}

type Customer implements User @key(fields: "id", resolvable: true) @shareable {
  id: ID!
  email: String
  name: String
}

type Query {
  product(id: ID!): Product @shareable
}

We have the main Product interface, and thee individual types that implement that interface (Computer, Monitor and Mouse). Each Product can have metadata values, that are interfaces that point to underlaying types.

Now, we have a very similar schema for product-new subgraph, and once again, we are leveraging @shareble so we can add both types in both subgrapahs (at this point of time, either subgraph can resovle the data)

extend schema
  @link(
    url: "https://specs.apollo.dev/federation/v2.8"
    import: [
      "@key"
      "@shareable"
      "@provides"
      "@external"
      "@tag"
      "@extends"
      "@override"
      "@inaccessible"
      "@composeDirective"
      "@interfaceObject"
    ]
  )

interface Product @key(fields: "id", resolvable: true) {
  id: ID!
  metadata: [ProductMetadataValue!]!
}

type Computer implements Product @key(fields: "id") {
  id: ID!
  metadata: [ProductMetadataValue!]! @shareable @override(from: "product-1")
}

type Monitor implements Product @key(fields: "id") {
  id: ID!
  metadata: [ProductMetadataValue!]! @shareable
}

type Mouse implements Product @key(fields: "id") {
  id: ID!
  metadata: [ProductMetadataValue!]! @shareable
}
interface ProductMetadataValue
  @key(fields: "_fieldId _productId", resolvable: true) {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  productMetadata: ProductMetadata!
}

interface ProductMetadata {
  id: ID!
  createdAt: String!
  createdBy: User
}

type TextProductValue implements ProductMetadataValue
  @key(fields: "_fieldId _productId")
  @shareable {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  textValue: String!
  productMetadata: TextProductMetadata!
}

type TextProductMetadata implements ProductMetadata @shareable {
  id: ID!
  createdAt: String!
  createdBy: User
}


type User @interfaceObject @key(fields: "id", resolvable: true) {
  id: ID!
}

More or less exactly the same. Only differences is, product-2 subgraph doesn't care about User implementations and uses interfaceObject, and main Product types don't include all the fields.

This works fine, and I get the expected behaviour

Problem is, when we start adding more of this ProductMetadata shared types... (we have more than 10! DropDown, Mutli and Single options, CreditCard, Dates, etc...)... After adding 6 of these types in both subgraphs, the Gateway code fails and blows up NodeJS


<--- Last few GCs --->

[52658:0x7fa930008000]   179699 ms: Scavenge 4010.4 (4111.7) -> 3995.7 (4097.6) MB, pooled: 0 MB, 5.09 / 0.00 ms  (average mu = 0.329, current mu = 0.502) allocation failure;
[52658:0x7fa930008000]   182387 ms: Mark-Compact 4011.7 (4112.9) -> 3943.3 (4045.1) MB, pooled: 0 MB, 2676.98 / 0.00 ms  (average mu = 0.243, current mu = 0.171) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Adding more than 10 produces a different error actually

{"code":"INTERNAL_SERVER_ERROR","stacktrace":["RangeError: Invalid array length","    at flatCartesianProduct (/app/node_modules/@apollo/query-graphs/dist/graphPath.js:1291:21)","    at advanceSimultaneousPathsWithOperation (/app/node_modules/@apollo/query-graphs/dist/graphPath.js:1245:24)","    at QueryPlanningTraversal.handleOpenBranch (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:151:96)","    at QueryPlanningTraversal.findBestPlan (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:136:18)","    at computeRootParallelBestPlan (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:2012:36)","    at computeRootParallelDependencyGraph (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:2008:12)","    at computePlanInternal (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:1945:33)","    at QueryPlanner.buildQueryPlan (/app/node_modules/@apollo/query-planner/dist/buildPlan.js:1822:24)","    at /app/node_modules/@apollo/gateway/dist/index.js:79:58","    at AsyncLocalStorage.run (node:async_hooks:346:14)"]}}],

And the GW returns a 500 with internal error server

Link to Reproduction

not yet

Reproduction Steps

Use this supergraph with latest version of Gateway:

schema
  @link(url: "https://specs.apollo.dev/link/v1.0")
  @link(url: "https://specs.apollo.dev/join/v0.5", for: EXECUTION)
  @link(url: "https://specs.apollo.dev/inaccessible/v0.2", for: SECURITY) {
  query: Query
}

directive @inaccessible on FIELD_DEFINITION | OBJECT | INTERFACE | UNION | ARGUMENT_DEFINITION | SCALAR | ENUM | ENUM_VALUE | INPUT_OBJECT | INPUT_FIELD_DEFINITION

directive @join__directive(
  graphs: [join__Graph!]
  name: String!
  args: join__DirectiveArguments
) repeatable on SCHEMA | OBJECT | INTERFACE | FIELD_DEFINITION

directive @join__enumValue(graph: join__Graph!) repeatable on ENUM_VALUE

directive @join__field(
  graph: join__Graph
  requires: join__FieldSet
  provides: join__FieldSet
  type: String
  external: Boolean
  override: String
  usedOverridden: Boolean
  overrideLabel: String
  contextArguments: [join__ContextArgument!]
) repeatable on FIELD_DEFINITION | INPUT_FIELD_DEFINITION

directive @join__graph(name: String!, url: String!) on ENUM_VALUE

directive @join__implements(
  graph: join__Graph!
  interface: String!
) repeatable on OBJECT | INTERFACE

directive @join__type(
  graph: join__Graph!
  key: join__FieldSet
  extension: Boolean! = false
  resolvable: Boolean! = true
  isInterfaceObject: Boolean! = false
) repeatable on OBJECT | INTERFACE | UNION | ENUM | INPUT_OBJECT | SCALAR

directive @join__unionMember(
  graph: join__Graph!
  member: String!
) repeatable on UNION

directive @link(
  url: String
  as: String
  for: link__Purpose
  import: [link__Import]
) repeatable on SCHEMA

type Agent implements User
  @join__implements(graph: PRODUCT_1_, interface: "User")
  @join__type(graph: PRODUCT_1_, key: "id", resolvable: true) {
  email: String
  id: ID!
  name: String
}

type BakAccountMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type BakAccountValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  iban: String!
  productMetadata: BakAccountMetadata!
}

type Computer implements Product
  @join__implements(graph: PRODUCT_1_, interface: "Product")
  @join__implements(graph: PRODUCT_2_, interface: "Product")
  @join__type(graph: PRODUCT_1_, key: "id")
  @join__type(graph: PRODUCT_2_, key: "id") {
  id: ID!
  description: String @join__field(graph: PRODUCT_1_)
  metadata: [ProductMetadataValue!]!
    @join__field(graph: PRODUCT_1_, usedOverridden: true)
    @join__field(graph: PRODUCT_2_, override: "product-1")
  cpu: String @join__field(graph: PRODUCT_1_)
}

type CreaditCardMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type CreaditCardValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  creditCardNumber: String!
  productMetadata: CreaditCardMetadata!
}

type Customer implements User
  @join__implements(graph: PRODUCT_1_, interface: "User")
  @join__type(graph: PRODUCT_1_, key: "id", resolvable: true) {
  id: ID!
  pepein: String
  email: String
  name: String
}

type DateMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type DateValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  date: String!
  productMetadata: DateMetadata!
}

type DropDownMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type DropDownValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  dropDownValue: String!
  productMetadata: DropDownMetadata!
}

input join__ContextArgument {
  name: String!
  type: String!
  context: String!
  selection: join__FieldValue!
}

scalar join__DirectiveArguments

scalar join__FieldSet

scalar join__FieldValue

enum join__Graph {
  PRODUCT_1_ @join__graph(name: "product-1", url: "http://alocalhost.com:3000")
  PRODUCT_2_ @join__graph(name: "product-2", url: "http://alocalhost.com:3000")
}

scalar link__Import

enum link__Purpose {
  """
  `SECURITY` features provide metadata necessary to securely resolve fields.
  """
  SECURITY

  """
  `EXECUTION` features provide metadata necessary for operation execution.
  """
  EXECUTION
}

type Monitor implements Product
  @join__implements(graph: PRODUCT_1_, interface: "Product")
  @join__implements(graph: PRODUCT_2_, interface: "Product")
  @join__type(graph: PRODUCT_1_, key: "id")
  @join__type(graph: PRODUCT_2_, key: "id") {
  id: ID!
  description: String @join__field(graph: PRODUCT_1_)
  metadata: [ProductMetadataValue!]!
  resolution: String @join__field(graph: PRODUCT_1_)
}

type Mouse implements Product
  @join__implements(graph: PRODUCT_1_, interface: "Product")
  @join__implements(graph: PRODUCT_2_, interface: "Product")
  @join__type(graph: PRODUCT_1_, key: "id")
  @join__type(graph: PRODUCT_2_, key: "id") {
  id: ID!
  description: String @join__field(graph: PRODUCT_1_)
  metadata: [ProductMetadataValue!]!
  dpi: String @join__field(graph: PRODUCT_1_)
}

type MultipleOptionMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type MultipleOptionValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  multipleOptionValue: String!
  productMetadata: MultipleOptionMetadata!
}

interface Product
  @join__type(graph: PRODUCT_1_, key: "id", resolvable: true)
  @join__type(graph: PRODUCT_2_, key: "id", resolvable: true) {
  id: ID!
  description: String @join__field(graph: PRODUCT_1_)
  metadata: [ProductMetadataValue!]!
}

interface ProductMetadata
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

interface ProductMetadataValue
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId", resolvable: true)
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId", resolvable: true) {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  productMetadata: ProductMetadata!
}

type Query @join__type(graph: PRODUCT_1_) @join__type(graph: PRODUCT_2_) {
  product(id: ID!): Product @join__field(graph: PRODUCT_1_)
}

type SingleOptionMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type SingleOptionValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  singleOptionValue: String!
  productMetadata: SingleOptionMetadata!
}

type TextProductMetadata implements ProductMetadata
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadata")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadata")
  @join__type(graph: PRODUCT_1_)
  @join__type(graph: PRODUCT_2_) {
  id: ID!
  createdAt: String!
  createdBy: User
}

type TextProductValue implements ProductMetadataValue
  @join__implements(graph: PRODUCT_1_, interface: "ProductMetadataValue")
  @join__implements(graph: PRODUCT_2_, interface: "ProductMetadataValue")
  @join__type(graph: PRODUCT_1_, key: "_fieldId _productId")
  @join__type(graph: PRODUCT_2_, key: "_fieldId _productId") {
  _fieldId: ID! @inaccessible
  _productId: ID! @inaccessible
  textValue: String!
  productMetadata: TextProductMetadata!
}

interface User
  @join__type(graph: PRODUCT_1_, key: "id", resolvable: true)
  @join__type(
    graph: PRODUCT_2_
    key: "id"
    resolvable: true
    isInterfaceObject: true
  ) {
  email: String @join__field(graph: PRODUCT_1_)
  id: ID!
}

Launch this query:

query TestProductMetadataValues($productId: ID!) {
      product(id: $productId) {
        id
        metadata {
          productMetadata {
              createdAt
          }
        }
      }
    }

with variable productId: 1234

the process will hang for ever and will eventually run out of memory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nogates