dynamic GraphQL server schema and babel-relay-plugin

[faassen]

Imagine a server that dynamically generates a GraphQL schema based on database content. The schema is only known during runtime, not compile time.

How does this interact with babel-relay-plugin? babel-relay-plugin requires a static dump of the server schema, but with a dynamic GraphQL schema that schema does not exist at this time. Recompiling the JS frontend code each time the server schema changes is not an option.

How would you make this work?

[cesarandreu]

I think there was a discussion around this in Reactiflux (and I thought in an issue as well, but I can't find it). The conclusion, as I recall, was basically to avoid it.

If there's stuff you want to avoid exposing to users (such as admin-only stuff), you could generate two schemas and compile the app with each. Otherwise you return null for stuff a user can't access. What prevents you from knowing the schema at compile-time?

[josephsavona]

Forgot compiling, how would you even write product code against a schema that is effectively unknown?

[faassen]

An architecture where the complete schema is only known at runtime is quite common in software in my experience. Take a CMS where users can create new types of content using a UI during runtime. Or take any random enterprise software system where types and their fields are defined in a database, again by users with an UI.

Not everybody is building single-deployment web applications. A lot of software has a multiple independent deployments, each with a different configuration and extensions. Some core types and fields will be in common between all of them, but there are also fields and entire types unique to a deployment. Each schema effectively has its own schema.

Whether this is the best way to architect a system is another debate: can we discuss this accepting that such systems already exist and that it may be interesting to use Relay with them? In some ways a database-generated schema makes software like this a great fit for GraphQL, as this kind of software typically already has sufficient information in its database to generate a schema.

There are multiple ways you can write UIs for software where you don't know the complete schema:

• the schema can be introspected during runtime, and a UI can in part be auto-generated. This may of course need supplementary information beyond what's in a GraphQL schema.
• a subset of the schema may be completely known during development time, even though the software makes you store its definition in the database due to legacy reasons -- that's how all schema information is stored.
• you may not know everything about concrete types in complete deployments, but you may know quite a lot about their interface, and thus you can write a generic UI that can handle these.

graphql-js seems, as far as I can see, to be explicitly engineered to allow the generation of schema information at runtime where needed. But babel-relay-plugin, whatever it does exactly (I don't quite understand yet), is a problem.

Can babel-relay-plugin be avoided entirely? What are the consequences of doing so?

@cesarandreu points at a possible approach: it may be possible to know a subset of the complete schema at compile time. But how would this work for new fields or complete types that only known during runtime?

@josephsavona I chatted with you briefly at React Conf Europe I believe, just before your Relay talk. So, are you asking me how people can write code in a dynamically typed language? I can try to explain... :)

[josephsavona]

@faassen I apologize if my question came off as snarky, but I was genuinely curious and want to further understand the problem space. What I was getting at with my question is that the code clearly has to have some understanding of the structure of the data it will receive (this isn't about static/dynamic languages). Let's take as example a simple CMS that allows users to define custom types, custom fields on those types, and provides UIs for editing objects of these types. It would seem that all product code in this system would fall into one of two categories:

• Meta-code. For example, a view that queried for the fields of a user-defined type and displays a dynamic form based on the (dynamically) defined form fields.
• Product-specific code. For example, a product-specific customization might assume the existence of a Foo type with a bar field, and show a tailored UI just for this type.

In both cases, the schema is actually knowable at build time. In the meta-code example the schema is the meta-schema - the schema the CMS framework uses to describe the custom types and fields. In the product-specific case, the product schema can be dynamically generated at build time to validate the code.

Overall, it's absolutely possible to build Relay apps using the type of system you're describing. The schema is required for compiling queries, but this can be dynamically generated by executing the introspection query against the server prior to the build step. This is actually how we compile Relay queries at Facebook - by fetching the introspection queries from our GraphQL server and caching the results on disk.

Can babel-relay-plugin be avoided entirely

No, because Relay needs information about the schema in order to process queries. However, as described above it should always be possible to get a schema such that queries can be compiled.

[josephsavona]

@faassen Did this address your question? It definitely seems that what you're looking for is possible, unless I'm missing something.

[faassen]

@josephsavno Thank you for the helpful answer!

I think meta code is a good way to understand the problem of dynamically generated UIs, and instead of a concrete schema we'd have a schema that describes the schema and build on that.

That leaves what you call "product-specific code". I think the question transforms into how one would create a composable GraphQL server.

So imagine you're working on an extension to a CMS that handles a particular type of content, say "customer". You can define the schema that deals with customer-related information as part of this extension. In it, you can also build on the schema defined by the core of the CMS, and possibly the schemas of other extensions that your extension depends on.

So, during development you could export this schema and use it to compile the JS code. But in reality this schema is only a subset of the complete subset available during deployment. What is available during deployment depends on what extensions get installed.

Is it possible to compile the code against a partial schema (a working subset of the full schema) or is the full schema necessary? Would recompilation be needed during deployment once the full schema is known or can this be avoided entirely? This depends a bit on what babel-relay-plugin actually does: if it's just about validating a query, then it sounds like recompilation after deployment can be avoided, but if it's also about intelligent cache invalidation, then perhaps it cannot.

In other words, can one develop code against a subset of a larger schema that only contains those aspects of the total schema that are in fact in use by the code?

[josephsavona]

In other words, can one develop code against a subset of a larger schema that only contains those aspects of the total schema that are in fact in use by the code?

Yes. So long as every field/type that your queries use are in the schema that you compile against, this should be fine. The plugin does a combination of validation (do those types/fields/arguments exist?) and annotation. Annotation includes marking fields as connections so that they can be processed as such, or ensuring that queries contain an id field if it exists in the schema.

[faassen]

Okay, that's good to know! So recompilation would not be needed after code composition time; this is something that one would like to avoid in some architectures. Thanks for helping to make this whole issue more clear in my head. I'll close this issue now.

If we ever end up implementing this we'll undoubtedly run into details but I'll ask those if and when needed.

[nodkz]

For future googlers:
@taion released awesome scalar GraphQLType JSON that supports dynamic values for a field!

I'm using it in graphql-compose-mongoose, so live demos with dynamic field's type can be found here:

• Find User with field of MIXED type
• Create user mutation (with arg of MIXED type)

[limscoder]

@nodkz Is there then a way to specify which fields within someMixed should be fetched and returned? My use case is that you don't want to return all of someMixed's fields, and someMixed fields are not known at build time.

[nodkz]

@limscoder field someMixed with type JSON consider by GraphQL as scalar value. There is no way to get partial value of such field. So JSON type is black box.

As solution may be used args for field. Depending on provided args for such field u resolve what u need.