When graphqlgen large schema will get stuck #310

Victorkangsh · 2018-11-23T04:01:29Z

Description

In one of my projects, the datamodel has five hundred lines. My schema.graphql file imports the generated prisma.graphql schema file and has a hundred lines of Query and Mutation.

I get stuck when I run graphqlgen. Cpu and memory are slightly higher. I noticed a prompt for the find prettier, but I still didn't succeed after removing the prettier configuration.

My three computers have linux, mac, and windows systems, and they are not working properly. But the surprise is that my mac has been successful once, just once.

Steps to reproduce

When schema.graphql too large.

My other small project run graphqlgen perfectly.

Expected results

Graphqlgen work success.

Actual results

Stucked. And console has no message.

Versions

graphqlgen: 0.4.0
OS name and version: mac: 16G memory, mojava(darwin-x64). windows: win10 16G memory i7.
Node: node-v10.8.0

The text was updated successfully, but these errors were encountered:

Victorkangsh · 2018-11-23T05:11:49Z

Supplement：
When wait 1 hour later, console log err as follows：

graphqlgen
<--- Last few GCs --->

[14832:000002002A721AF0]  8948164 ms: Mark-sweep 1159.4 (1202.0) -> 1159.4 (1165.5) MB, 3679.4 / 0.0 ms  last resort GC in old space requested
[14832:000002002A721AF0]  8948951 ms: Mark-sweep 1159.4 (1165.5) -> 1159.4 (1165.5) MB, 786.8 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000038B35425EE1 <JSObject>
    0: builtin exit frame: concat(this=000002C19A6AE511 <JSArray[40009211]>,000002C19A682201 <JSArray[18728217]>)

    1: flatten(aka flatten) [C:\Users\T47\AppData\Roaming\npm\node_modules\graphqlgen\dist\utils.js:~49] [pc=000002BC79E8D883](this=0000006577382311 <undefined>,a=000002C19A6AE511 <JSArray[40009211]>,b=000002C19A682201 <JSArray[18728217]>)
    2: arguments adaptor frame: 4->2
    ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

And when I set --max_old_space_size setting it still not working.

LawJolla · 2018-11-25T16:54:49Z

Same issue. Looks like it's in a loop of some kind. I am importing one type from Prisma via graphql-import.

MacOS 10.14.1, 2015 iMac with 16gb RAM and 2016 MBP with 8gb RAM

<--- Last few GCs --->

[1153:0x104001c00]   403737 ms: Mark-sweep 1219.6 (1227.3) -> 1219.6 (1226.3) MB, 736.6 / 0.0 ms  (average mu = 0.079, current mu = 0.000) allocation failure GC in old space requested
[1153:0x104001c00]   404472 ms: Mark-sweep 1219.6 (1226.3) -> 1219.6 (1225.3) MB, 735.3 / 0.0 ms  (average mu = 0.052, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x05326251d9f1 <JSObject>
    0: builtin exit frame: concat(this=0x0532c9e86071 <JSArray[15602824]>,0x0532e3b266a1 <JSArray[14814181]>,0x0532c9e86071 <JSArray[15602824]>)

    1: /* anonymous */(aka /* anonymous */) [0x53221beec69] [/Users/dennismacbookpro/WebstormProjects/wk-mono-repo/node_modules/graphqlgen/dist/generators/common.js:~142] [pc=0x19d5dd850c51](this=0x0532387825b1 <undefined>,0x05328ef92ef9 <String[15]: S...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10005eeab node::Abort() [/usr/local/bin/node]
 2: 0x10005f68a node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
 3: 0x100172027 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100171fc8 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x100433768 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x100435185 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x100432afd v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x1004318f3 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x100431b8f v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason) [/usr/local/bin/node]
10: 0x100439835 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
11: 0x1004195b3 v8::internal::Factory::AllocateRawArray(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
12: 0x100419469 v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::Heap::RootListIndex, int, v8::internal::Object*, v8::internal::PretenureFlag) [/usr/local/bin/node]
13: 0x100424f5a v8::internal::Factory::NewJSArrayStorage(v8::internal::Handle<v8::internal::JSArray>, int, int, v8::internal::ArrayStorageAllocationMode) [/usr/local/bin/node]
14: 0x100424e2b v8::internal::Factory::NewJSArray(v8::internal::ElementsKind, int, int, v8::internal::ArrayStorageAllocationMode, v8::internal::PretenureFlag) [/usr/local/bin/node]
15: 0x1003ce3ee v8::internal::ElementsAccessor::Concat(v8::internal::Isolate*, v8::internal::Arguments*, unsigned int, unsigned int) [/usr/local/bin/node]
16: 0x1001d9220 v8::internal::(anonymous namespace)::Fast_ArrayConcat(v8::internal::Isolate*, v8::internal::BuiltinArguments*) [/usr/local/bin/node]
17: 0x1001d6d5a v8::internal::Builtin_Impl_ArrayConcat(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/usr/local/bin/node]
18: 0x19d5dd7cfcdd 
19: 0x19d5dd850c51 
20: 0x19d5dd7876a6 
21: 0x19d5dd871a16 
[1]    1153 abort      graphqlgen

rowlandekemezie · 2018-11-28T19:25:15Z

@Victorkangsh Did you try downgrading to 0.3.0. Mine got stuck without any info tho' but works fine with 0.3.0

LawJolla · 2018-11-28T21:06:49Z

@mentrie Thanks for the suggestion. But if I downgrade, then I'm hit with the #280 bug 😢

rowlandekemezie · 2018-11-28T21:40:36Z

@LawJolla Right. For the past 1hr, 0.4.0 is still running 😢

frankdugan3 · 2018-12-07T18:02:45Z

Same issue here. Can't use at all until this is resolved. My system has 32 GB of RAM, and it still doesn't work. Haven't let it run for an hour, but after several minutes I still have plenty of RAM left.

woss · 2018-12-13T17:07:15Z

I've spend few hours debugging the package and it seems that here :
https://github.com/prisma/graphqlgen/blob/03336f01d1257643c4b24577a397df44bd00076a/packages/graphqlgen/src/generators/common.ts#L228 it turns into the endless loop.

When i add Mutation to my schema with the same name as one in the prisma.graphql it runs until it fails.

woss · 2018-12-13T17:08:14Z

@mentrie Thanks for the suggestion. But if I downgrade, then I'm hit with the #280 bug 😢

that has been fixed. I've downgraded to 0.3.0 and it worked even with the schema that fails on 0.4.0

LawJolla · 2018-12-13T17:55:33Z

@mentrie Thanks for the suggestion. But if I downgrade, then I'm hit with the #280 bug 😢

that has been fixed. I've downgraded to 0.3.0 and it worked even with the schema that fails on 0.4.0

@woss It's still an issue for me in 0.3.0. This project looked promising, but it now appears under maintained if not abandoned. I've given up and gone back to Javascript to actually get things built... pesky deadlines. :)

jasonkuhrt · 2018-12-19T01:48:15Z

Anyone have a schema that leads to this issue that they can share?

Victorkangsh · 2018-12-19T05:02:58Z

@jasonkuhrt I have figured out its problem, it has nothing to do with the size of the schema, but an error when using the imported Input Type.
like this example : https://github.com/prisma/graphql-prisma-typescript/blob/master/src/schema.graphql

when you add some Mutation using imported Input Type it will stuck:

# import * from "./generated/prisma.graphql"
type Mutation {
  addPaymentMethod(
    data: PaymentCreateInput
  ): MutationResult!
  book(
    data: BookingCreateInput
  ): MutationResult!
}

vdiaz1130 · 2018-12-20T00:44:15Z

This bug renders graphqlgen unusable for me. Running with version 3 I get a bunch of TS errors while running on latest gets stuck.

I was more specific with my imports from Prisma generated code.

// I used
# import Post, User, UserCreateInput, UserUpdateInput from './generated/prisma.graphql'

// VS
# import * from "./generated/prisma.graphql"

node -v
v11.4.0

npm -v
6.4.1

OSx 10.12.6

prisma -v
prisma/1.22.2

https://ibb.co/0DjYqkG

Any hacky workaround ideas for now? A nightly I can test?

vdiaz1130 · 2018-12-20T01:15:50Z

Quick update: in VS code, using cmd + . then selecting add all missing imports adds all missing imports - who would've thought?

jasonkuhrt · 2018-12-20T01:31:32Z

@Victorkangsh iiuc:

This issue occurs in conjunction with graphql import
I should be able to repro with
https://github.com/prisma/graphql-prisma-typescript/blob/master/src/schema.graphql
But a more minimal repro can be had by importing one schema (A) into another (B) and using A types in B Mutation fields args.

Did I get that right?

Victorkangsh · 2018-12-20T07:32:54Z

@jasonkuhrt I think the problem is nested InputType.
I use this config:

when I try this:

it's not work, and i copy in schema:

still stuck.
But when I remove nest Type , it worked:

Victorkangsh · 2018-12-20T07:38:36Z

graphql import has no err , I test this:

all success

vdiaz1130 · 2018-12-20T13:31:45Z

@jasonkuhrt

Anyone have a schema that leads to this issue that they can share?

https://github.com/vdiaz1130/graphqlgen-issue310

I copied over Prisma generated stuff and the schema (you wont be able to regen Prisma unless you configure all of that).

psycura · 2018-12-20T14:15:10Z

Anyone have a schema that leads to this issue that they can share?

gist

atardadi · 2018-12-20T15:56:09Z

To provide some more context.
I'm getting a runtime error after ~1hr from flatten method in utils.js.
The error is a Range Error

Added console.log to the array and the length does grow exponentially.

function flatten(a, b) {
    a.length > 100000 && console.log('length', a.length);

    return a.concat(b);
}

I could see that the type it's trying to parse is Mutation

Would highly appreciate your help with this one

alexbudure · 2018-12-22T20:27:39Z

@mentrie Thanks for the suggestion. But if I downgrade, then I'm hit with the #280 bug 😢

Experiencing the same issue. Any updates on this?

vdiaz1130 · 2018-12-27T15:20:14Z

I worked around this by not importing from Prisma generated. I simply copied over my schema and removed all the Prisma specific things (@relation, @default, etc). Also using version 0.3.0 of graphqlgen.

If in your Prisma generated code you have an error on the Model interface, delete the import and reimported via VS code "cmd + ." It should import from a ''prisma-client-lib/dist/types' - Fix this before you generate with graphqlgen.

After generating with graphqlgen, If you get missing imports from your generated code and you are using VS Code, go next to a type, class, interface etc, then hit "cmd + ." Select "add all missing imports".

Relates to #310

jasonkuhrt · 2019-01-01T13:26:05Z

Before considering this issue resolved I'd like to hear from multiple users if its their experience that the issue is resolved too.

LawJolla · 2019-01-01T19:18:00Z

Thanks @jasonkuhrt !

Is there a particular release we should try? I still see 0.4.0 as the current.

atardadi · 2019-01-01T21:09:46Z

Before considering this issue resolved I'd like to hear from multiple users if its their experience that the issue is resolved too.

I added the line you added in the pull request to my local code.
It's been running for ~90 mins without stopping.
Looks like it's still reproducing.

jasonkuhrt · 2019-01-02T02:08:36Z

@LawJolla I'm not doing releases yet myself I'm afraid (might start to this or next month though) but maybe you could pull down from git repo (but build steps and all...). Given the diff is trivial what @atardadi did might be easiest.

@atardadi thanks for the feedback! Good to know...

-- Update

Yeah, this issue is not resolved I'm pretty sure.

I just pushed a commit that increased performance of the surrounding function by 6x (effect measured and tracked in benchmark history). It's just a microbenchmark improvement. The existing integration benchmarks did not improve.

Sorry for false hope, still in the woods. I am however continuing to get a better grasp of codebase. Hopefully we'll close this soon!

jasonkuhrt · 2019-01-02T02:15:48Z

@vdiaz1130 thanks for the repo repro. I'd like to extract the minimal:

model file(s)
schema file(s)

needed, and repro with the graphqlgen test suite.

But, can you advise how I should get prisma-client-lib installed and setup exactly? Even better might be a repro that doesn't need that dep to begin with (would simplify a bit I suppose).

Victorkangsh · 2019-01-02T10:24:55Z

My project can be generated by #373
Only the last problem left: #278

jasonkuhrt · 2019-01-03T02:19:55Z

@Victorkangsh ok thanks for confirming, interesting... I think then, since this issue was originally created by you we should close it. Others have raised other issues here but going by your feedback, they are different issues. As such I would like to see them tracked as separate issues and have focused discussion.

Does anyone strongly object?

atardadi · 2019-01-03T15:01:34Z

@Victorkangsh ok thanks for confirming, interesting... I think then, since this issue was originally created by you we should close it. Others have raised other issues here but going by your feedback, they are different issues. As such I would like to see them tracked as separate issues and have focused discussion.

Does anyone strongly object?

I don't mind, although I have the same issue - once my schema grew large it doesn't generate it for me.
We could try publishing a new version and see if that solves it for me.
Otherwise, I will create a separate issue

jasonkuhrt · 2019-01-03T16:04:03Z

We could try publishing a new version and see if that solves it for me.

I haven't been doing that personally yet, though I think making a pre-release would be fine without much prep, however I don't think I have the required npm rights yet.

delianides · 2019-01-08T18:02:21Z

@Victorkangsh was there a separate issue created for this?

schickling added bug/2-confirmed status/pr-welcome labels Nov 28, 2018

jasonkuhrt self-assigned this Dec 19, 2018

zazido mentioned this issue Dec 27, 2018

Fix input types resolver in generators #373

Merged

1 task

jasonkuhrt mentioned this issue Dec 28, 2018

perf: introduce benchmarks #372

Merged

7 tasks

jasonkuhrt closed this as completed in #373 Jan 1, 2019

jasonkuhrt pushed a commit that referenced this issue Jan 1, 2019

perf(core): do not deeply resolve an input type more than once (#373)

ae1b596

Relates to #310

jasonkuhrt reopened this Jan 1, 2019

jasonkuhrt removed the status/pr-welcome label Jan 1, 2019

Victorkangsh closed this as completed Jan 4, 2019

devautor mentioned this issue Feb 11, 2019

graphqlgen gives UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'map' of undefined #449

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When graphqlgen large schema will get stuck #310

When graphqlgen large schema will get stuck #310

Victorkangsh commented Nov 23, 2018

Victorkangsh commented Nov 23, 2018 •

edited

Loading

LawJolla commented Nov 25, 2018 •

edited

Loading

rowlandekemezie commented Nov 28, 2018 •

edited

Loading

LawJolla commented Nov 28, 2018 •

edited

Loading

rowlandekemezie commented Nov 28, 2018

frankdugan3 commented Dec 7, 2018

woss commented Dec 13, 2018

woss commented Dec 13, 2018

LawJolla commented Dec 13, 2018 •

edited

Loading

jasonkuhrt commented Dec 19, 2018

Victorkangsh commented Dec 19, 2018 •

edited

Loading

vdiaz1130 commented Dec 20, 2018

vdiaz1130 commented Dec 20, 2018

jasonkuhrt commented Dec 20, 2018

Victorkangsh commented Dec 20, 2018

Victorkangsh commented Dec 20, 2018 •

edited

Loading

vdiaz1130 commented Dec 20, 2018 •

edited

Loading

psycura commented Dec 20, 2018 •

edited

Loading

atardadi commented Dec 20, 2018 •

edited

Loading

alexbudure commented Dec 22, 2018

vdiaz1130 commented Dec 27, 2018

jasonkuhrt commented Jan 1, 2019

LawJolla commented Jan 1, 2019

atardadi commented Jan 1, 2019

jasonkuhrt commented Jan 2, 2019 •

edited

Loading

jasonkuhrt commented Jan 2, 2019 •

edited

Loading

Victorkangsh commented Jan 2, 2019

jasonkuhrt commented Jan 3, 2019

atardadi commented Jan 3, 2019

jasonkuhrt commented Jan 3, 2019

delianides commented Jan 8, 2019

When graphqlgen large schema will get stuck #310

When graphqlgen large schema will get stuck #310

Comments

Victorkangsh commented Nov 23, 2018

Description

Steps to reproduce

Expected results

Actual results

Versions

Victorkangsh commented Nov 23, 2018 • edited Loading

LawJolla commented Nov 25, 2018 • edited Loading

rowlandekemezie commented Nov 28, 2018 • edited Loading

LawJolla commented Nov 28, 2018 • edited Loading

rowlandekemezie commented Nov 28, 2018

frankdugan3 commented Dec 7, 2018

woss commented Dec 13, 2018

woss commented Dec 13, 2018

LawJolla commented Dec 13, 2018 • edited Loading

jasonkuhrt commented Dec 19, 2018

Victorkangsh commented Dec 19, 2018 • edited Loading

vdiaz1130 commented Dec 20, 2018

vdiaz1130 commented Dec 20, 2018

jasonkuhrt commented Dec 20, 2018

Victorkangsh commented Dec 20, 2018

Victorkangsh commented Dec 20, 2018 • edited Loading

vdiaz1130 commented Dec 20, 2018 • edited Loading

psycura commented Dec 20, 2018 • edited Loading

atardadi commented Dec 20, 2018 • edited Loading

alexbudure commented Dec 22, 2018

vdiaz1130 commented Dec 27, 2018

jasonkuhrt commented Jan 1, 2019

LawJolla commented Jan 1, 2019

atardadi commented Jan 1, 2019

jasonkuhrt commented Jan 2, 2019 • edited Loading

jasonkuhrt commented Jan 2, 2019 • edited Loading

Victorkangsh commented Jan 2, 2019

jasonkuhrt commented Jan 3, 2019

atardadi commented Jan 3, 2019

jasonkuhrt commented Jan 3, 2019

delianides commented Jan 8, 2019

Victorkangsh commented Nov 23, 2018 •

edited

Loading

LawJolla commented Nov 25, 2018 •

edited

Loading

rowlandekemezie commented Nov 28, 2018 •

edited

Loading

LawJolla commented Nov 28, 2018 •

edited

Loading

LawJolla commented Dec 13, 2018 •

edited

Loading

Victorkangsh commented Dec 19, 2018 •

edited

Loading

Victorkangsh commented Dec 20, 2018 •

edited

Loading

vdiaz1130 commented Dec 20, 2018 •

edited

Loading

psycura commented Dec 20, 2018 •

edited

Loading

atardadi commented Dec 20, 2018 •

edited

Loading

jasonkuhrt commented Jan 2, 2019 •

edited

Loading

jasonkuhrt commented Jan 2, 2019 •

edited

Loading