Spec: Formal Subscriptions Definition #305

robzhu · 2017-05-02T21:17:38Z

This PR contains my proposal for formally describing GraphQL Subscriptions in the Spec. It will be accompanied by a PR against GraphQL-js (will provide link when that PR is available).

wincent · 2017-05-02T23:39:42Z

spec/Section 5 -- Validation.md

+    body
+    sender
+  }
+  # a second root field will cause this operation to fail validation


Everywhere else in this file we rely on naming rather than comments to illustrate counterexamples. Perhaps we should do the same for consistency here. (ie. call the field disallowedSecondRootField)?

wincent · 2017-05-02T23:40:36Z

spec/Section 5 -- Validation.md

+    sender
+  }
+  # this also applies to introspection fields
+  __typename


Not clear what we would name this one if we wanted to get rid of the comment.

Added a description above.

wincent · 2017-05-02T23:43:06Z

spec/Section 6 -- Execution.md

@@ -143,6 +143,84 @@ ExecuteMutation(mutation, schema, variableValues, initialValue):
    selection set.
  * Return an unordered map containing {data} and {errors}.


Putting a comment here because silly GitHub won't let me comment on the lines above. Up on line 106-107 it says:

If mutations are supported, it must also provide a mutation root object type.

That should probably say:

If mutations or subscriptions are supported, it must also provide a mutation and subscription root object type, respectively.

Or words to that effect.

Also noticed that a step should be added to {ExecuteRequest} above to handle subscription requests.

wincent · 2017-05-02T23:44:51Z

spec/Section 6 -- Execution.md

+phases. Between the the subscribe and unsubscribe phases, the subscription is
+considered to be in the "active" state.
+
+### Subscribe


We should probably add new subheadings to this section:

## Executing Operations ... ### Executing Queries ... ### Executing Mutations ... ### Executing Subscriptions ... #### Subscribe (etc)

wincent · 2017-05-02T23:45:42Z

spec/Section 6 -- Execution.md

+  * **must** support cancellation of the subscription (aka unsubscribe).
+  * **must** support a way to identify the subscriber/subscription pair (for
+    example, a GUID/callback table or closure over the callback).
+  * **may** include an initial response associated with executing the selection


Capitalize "m" in "must"/"may" here for consistency with the other lists (that all seem to start with capitals)?

wincent · 2017-05-02T23:46:29Z

spec/Section 6 -- Execution.md

+
+  * **must** support observation of the associated publish stream (for example,
+  via iteration, callbacks, or reactive semantics).
+  * **must** support cancellation of the subscription (aka unsubscribe).


"aka" is a bit informal: I'd suggest, "that is," instead.

wincent · 2017-05-02T23:48:29Z

spec/Section 6 -- Execution.md

+the following capabilities:
+
+  * **must** support observation of the associated publish stream (for example,
+  via iteration, callbacks, or reactive semantics).


Not sure what "iteration" means in the context of observation: do you specifically mean async iteration? (and if so, any concern that that is allowing JS-specific terminology to leak into the spec?)

I was referring to the general iterator pattern (async iterators also count). For example, Unity handles async operations via iterators/generators, so I don't think it's JS-specific.

wincent · 2017-05-02T23:50:33Z

spec/Section 6 -- Execution.md

+  * **may** include an initial response associated with executing the selection
+  set defined on the subscription operation.
+
+Subscribe(subscription, schema, variableValues, initialValue):


initialValue is not referenced anywhere in this section.

good catch, this is a good place to talk about returning an initial response.

wincent · 2017-05-02T23:52:39Z

spec/Section 6 -- Execution.md

+
+Publish(subscription, schema, variableValues, payload, publishStream)
+
+  * For each {eventStream} as {event} and {payload}:


Where does payload come from? On line 169 above it says that eventStream is "the result" of running MapSubscriptionEvents, but the "result" isn't defined anywhere.

Rewrote this section to make it more clear.

wincent · 2017-05-02T23:53:48Z

spec/Section 6 -- Execution.md

+      *normally* (allowing parallelization).
+    * Let {errors} be any *field errors* produced while executing the
+      selection set.
+    * If {errors} is not empty, optionally terminate the subscription.


Should we define what termination means?

changed "Unsubscribe"

wincent · 2017-05-02T23:55:22Z

spec/Section 6 -- Execution.md

+
+Unsubscribe()
+
+  * Let {publishStream} be a mapping of {null}


I'm not sure what this means.

reworded this to "Terminate {publishStream}" with some examples.

wincent · 2017-05-02T23:55:56Z

spec/Section 6 -- Execution.md

+  * Subscriber/client channel
+  * Subscription document
+  * Variables
+  * Execution context (for example, current logged in user, locale, etc.)


s/logged in/logged-in/

wincent · 2017-05-02T23:56:41Z

spec/Section 6 -- Execution.md

+
+Rather than mixing stateless (queries and mutations) and stateful
+(subscriptions), we recommend keeping the GraphQL server stateless and
+delegating all state persistence these sub-systems.


these sub-systems → to these sub-systems

DxCx · 2017-05-03T07:47:45Z

spec/Section 5 -- Validation.md

+
+**Explanatory Text**
+
+Subscription operations must have exactly one root field.


@robzhu what is the reason for that? why can't i subscribe to more then one subscription with one call?
so far i thought this is implementation limitation..

I agree with Rob's text that starting with just one root field is reasonable. A later version of the spec could support multiple fields, but that requires figuring out a lot of edge cases around errors, when subfields are re-executed, and more.

I also would like to know the reason for this rule. I find that it is quite hard limitation, and in a way it goes against GraphQL principles where clients have the power to decide what it wants to get.

that requires figuring out a lot of edge cases around errors

I think it would be helpful to list and discuss these edge cases. From my experience, there are definitely things to consider when merging different event streams from different GraphQL subscription fields, but I find it quite manageable.

I listed most of the points I found as I was implementing it in sangria here:

#282 (comment)

I think that it boils down to points 3.i, 5 and 6. My take on it is here: #282 (comment)

IMHO, if we need to make a trade off, I would rather disallow not-null root subscription field types than allow only a single subscription field in a query.

@stubailo if one wants to experiment that's fine, but i don't think it should be in the official spec unless there is a reason (which is not implementation) behind it.

@stubailo thanks a lot for describing in on an example!

So you're sending two subscription fields in the request, but you're never going to get that entire response.

I don't see it as a problem but rather as a natural behavior. This is inherent property of event-based interaction. This is also the reason why I suggested to disallow not-null root subscription field types.

you can now only unsubscribe to the whole thing at once (maybe a benefit)

I also don't see it as a issue either. Can you describe in more detail why this behavior can be disadvantageous? (considering that you still can make 2 separate and isolated subscription queries if it suits better for the use-case at hand)

it's not clear what to do when one of them has a fatal error - do both get unsubscribed?

I feel that either behavior is fine as long as it is defined in the spec. Though in this case I would suggest draw inspiration from streaming libraries: if 2 event streams are joined/merged together in a single result stream, then an error in either of these will also case the result stream to fail. If one of event steams naturally completes (because of the exhaustion), then the result steam still continue to emit events until all of the source streams are exhausted. I think this behavior is quite intuitive and widespread.

@OlegIlyenko @DxCx To give a little bit of context on where this came from, we discovered early on that it was better to use subscriptions for modeling granular events. For example, consider the three main subscriptions that operate on a facebook post: live comments, live likes, and typing indicators. These are individual subscriptions as opposed to a single "postLiveUpdate" subscription. Keeping these subscriptions granular on the client made natural sense. @laneyk and @dschafer may be able to add more perspective here.

Thinking this through, if we include multiple root fields like so:

subscription sub(...) { liveLike (...) {...} likeComment (...) {...} }

So far, everyone seems to assume this subscription should publish data when either "live like" or "live comment" publishes. Is that clearly the intent of this query? What if there were a desire to trigger the publish only when both root fields have a publish payload available? How would we describe that? By limiting the selection to a single root field, we sidestep all that.

I also don't think the single-root-field-rule introduces any practical limitations to the client. In fact, it results in simpler client-side code, like so:

likeSubscription.subscribe(payload => updateLikeState(payload));

For subscriptions containing more than one root field, if we assume the "or" behavior, as @stubailo points out, you'll never have more than one event trigger at a time, so the code would end up looking like:

genericSubscription.subscribe(payload => { if (payload.subscriptionA) { updateA(payload.SubscriptionA); } else if (payload.subscriptionB) { updateB(payload.SubscriptionB); } // etc. });

Can someone help me understand a compelling use case that is served by multi-root subscription operations that would not be equally served by separate individual subscriptions?

Just to add a meta-point to this conversation:

In my view, there's nothing stopping us from working through these details and figuring out the edge and corner cases of allowing multiple subscription fields in a single request, however the choice we do have is to address those concerns now, or allow for more time to do so. In previous conversations @robzhu has had over the last few months about subscriptions, he has convinced many that this is far more complicated than we originally thought and may not have clear answers. This limitation is added mostly in a desire to expedite the addition of subscriptions to the spec, while reserving the ability to continue to work out how or if multi-field subscriptions should be allowed.

Had this limitations been omitted while also not making mention of how to address multi-field subscriptions in the spec, then we would see divergence of behavior and that could tie our hands in the future for deciding how to address these edge cases.

Thanks a lot @robzhu and @leebyron for the insightful comments! I think now I got a better understanding of the issue. I was also in two minds on this. On one hand, I wanted to get a better understanding about motivation behind inclusion of this rule. But on the other hand, I don't want to delay the progress on subscriptions incision in the spec. I tend to agree that it is a good idea to disallow multiple fields for now and start a separate discussion. I think it is a discussion worth having. I am actually very glad that this point is considered in the spec since I was also quite concerned about the semantics of multiple subscriptions fields.

Can someone help me understand a compelling use case that is served by multi-root subscription operations that would not be equally served by separate individual subscriptions?

In general, I found it very valuable to have as much information from client as possible in single query. For example, the fact that a client can express its requirements for a view or particular part of the application in a single query allowed us to make very interesting optimizations which would be quite hard to do otherwise (it is quite hard to correlate seemingly independent requests/queries). So by allowing client to better express it's requirements with several subscription fields in a single query, we open a door for potential server-side optimizations.

Now that I'm equipped with new insights, I will give it another thought. This thread was definitely helpful in this respect.

Before we will introduce this rule though, I think it is important to consider the nullability of the subscription fields, as i mentioned above. It is possible to make a nullable field not-null later on in a backwards-compatible way. If we allow subscription fields to be not-null now, it might become a challenge in future to allow multiple subscription fields, if we decide to do so.

Nullability may mean two different things in this context - this is a great point we should address.

One thing it may mean is that a subscription may not exist given some inputs in a way that isn't considered an error. I think this interpretation is both not what you were referring to, and also probably confusing to think about. The schema talks about the type of the payload result - so we're talking about the types of responses. We should probably make this point in the spec to clarify.

Secondly the nullability of the responses. This is one of the concerns with multi-field subscriptions to address later. For example, should it be legal to have a subscription field streamThings: String? where it is legal for any payload in the event sequence to in fact be null? I don't see a compelling reason to explicitly disallow this - though it is an edge case.

I think handling the payloads of multi-field subscriptions will need to account for this

laneyk · 2017-05-03T16:00:51Z

spec/Section 6 -- Execution.md

+  * **Must** support observation of the associated publish stream (for example,
+  via iteration, callbacks, or reactive semantics).
+  * **Must** support cancellation of the subscription, see {Unsubscribe}.
+  * **Must** support a way to identify the subscriber/subscription pair (for


I'm confused by what this means even with the additional explanation. What do you mean by "identify the subscriber/subscription pair"?

Updated to say "Must support a way to send data to the subscriber."

laneyk · 2017-05-03T16:03:22Z

spec/Section 6 -- Execution.md

+event carries an optional payload, which we combine with the arguments from
+Subscribe() to resolve the selection set.
+
+  * For each {event} and {payload} on {eventStream}:


Weren't we going to stop calling this "payload"?

Renamed this "eventData"

laneyk · 2017-05-03T16:05:06Z

spec/Section 6 -- Execution.md

+  * Let {selectionSet} be the top level Selection Set in {subscription}.
+  * Let {rootField} be the first top level field in {selectionSet}.
+  * Let {eventStream} be the result of running {MapSubscriptionEvents(rootField, variableValues)}.
+  * Optionally: let {data} be the result of running {ExecuteRequest(schema, document, operationName, variableValues, initialValue)}, and return {data} on the subscription object.


What is initialValue here? What's the value of specifying here and above (lines 166-7) that we can have an initial result? This leads to a lot of questions about how the execution works without any event that we don't address anywhere. Or is there discussion of this elsewhere?

Based on this discussion #283, if someone builds subscriptions that can be evaluated without event data, we would not necessarily want that to be in violation of the spec. I've added more text above to clarify the reasoning.

I find this very confusing and probably not what we want. If I understand this correctly, sometimes this algorithm results in a value {data} which is returned, sometimes it does not - in which case it returns nothing?

Also, this algorithm does not describe what should be done with {eventStream} once created - I would have expected {eventStream} to be returned.

As a concrete example - what should I expect to see differently between a subscription which has an initial response vs one that does not, say in terms of websocket payloads? Do I actually get a {} or {data: null} payload as an "ACK" for subscriptions without an initial value? Or is "ACK" a detail that belongs to the transport mechanism?

Perhaps another way to frame this is that "initial value" could be a first published event which simply occurs instantaneously after the creation of a subscription?

That's valid feedback, but I think merging the initial value into the publish stream might weaken the value proposition here. Let me try to give some context.

Chat room application: we want to subscribe to when users log on or off to keep the list of members in a chatroom up-to-date. But the subscription doesn't give us the set of initial users in the room. So we issue a query for all users in current chat room, and then fire up the subscription. But now we have a small race condition between when we fetched the chat participant list and when the subscription is created where we are no longer "basing" off the right state. It's even possible that the subscription sends its first payload before the query returns. The client-side code to handle this kind of scenario is tricky. Can we do better?

Returning an "initial value" is an attempt to address these sorts of scenarios: in other words, to create a sort of atomic hybrid query/subscribe. @taion, please chime in if I've misrepresented your requirements.

I had proposed that the "initial value" can be tacked onto the "ACK" that comes with successful subscription creation, but there was some push back in the discussion. If it's not part of the ACK, then it will need another protocol message type, with the guarantee that if the initial response is sent at all, it will precede any publish payloads.

If we merge simply set the initial value as the first publish payload, then we'll end up with something like this (from @ rmosolgo):

subscription { commentCreated(postId: 4) { # `null` on first request, but present in later updates: comment { body } # present on first request, but `null` in later updates: initialPost { title, comments { body } } } }

Maybe this is fine. I wanted to mention that returning initial value independent of the publish stream would not violate the spec. But you're raising some good questions. Definitely open to suggestions here.

I had (and still have) the same reaction as @leebyron to this part of the spec. I find it confusing and I don't really understand the value we're adding by explicitly mentioning it. As I mentioned before, this option raises a lot of questions for me that we don't really answer elsewhere. By calling out this "atomic hybrid query/subscribe" operation here, we're making it seem like a central concept for subscriptions (which I don't think it is). What if we leave this out of the spec until we can get more data on the demand for it and the best way to do it?

@robzhu and I talked about this offline, but I just wanted to point out that this initialValue addition improves but doesn't guarantee a solution to race conditions. Events can still potentially be repeated or dropped between the generation of the initial value payload and subsequent payloads, depending on the system design and the latency of events flowing through it.

I think the kicker here is that most examples of wanting initial data before subscription payloads desire a different query shape - in the example above a list of comments vs an individual comment. The spec as written wouldn't allow for this distinction, so I actually think your example is more powerful while requiring less from the spec.

Upon further discussion with @laneyk and @leebyron, initial value behavior can be supported out-of-the-box using two sub-fields in a subscription, as in @rmosolgo's example. However, the model of an "atomic query/subscribe" is probably better supported via batching. As a result, my next edit will remove any mention of initial value.

To be clear, the resulting edit is that we will simply not mention server behavior with respect to initial value. If you choose to implement/use it, that does not make your server violate the spec.

stubailo

This looks really great, just a few questions and suggestions.

Do you think it would be helpful to include a diagram? I think the RFC content was very helpful to give people a sense of what the text was trying to say, and I don't know if people can really construct an image of what is going on from this spec. Perhaps that's the job of the documentation, though.

stubailo · 2017-05-08T07:10:16Z

spec/Section 5 -- Validation.md

+
+**Explanatory Text**
+
+Subscription operations must have exactly one root field.


I agree with Rob's text that starting with just one root field is reasonable. A later version of the spec could support multiple fields, but that requires figuring out a lot of edge cases around errors, when subfields are re-executed, and more.

stubailo · 2017-05-08T07:11:33Z

spec/Section 6 -- Execution.md

-provide a query root object type. If mutations are supported, it must also
-provide a mutation root object type.
+provide a query root object type. If mutations/subscriptions are supported, it must also
+provide a mutation/subscription root object type, respectively.


Probably should be plural:

provide mutation/subscription root object types, respectively

However, I'd be in favor of splitting into two sentences, one about mutations and one about subscriptions.

stubailo · 2017-05-08T07:12:33Z

spec/Section 6 -- Execution.md

@@ -143,6 +147,96 @@ ExecuteMutation(mutation, schema, variableValues, initialValue):
    selection set.
  * Return an unordered map containing {data} and {errors}.

+### Subscription
+
+Unlike queries and mutations, subscriptions have a lifetime that consists of three


If this sentence is going to start with "unlike", it should probably provide the thing we are comparing to, like:

Unlike queries and mutations, which deal with one request and response, subscriptions have a lifetime that consists of three

It's comparing Subscriptions to the other two operation types, so I don't think an example is necessary. I'll reword so that the sentence no longer starts with "unlike".

stubailo · 2017-05-08T07:13:50Z

spec/Section 6 -- Execution.md

+  via iteration, callbacks, or reactive semantics).
+  * **Must** support cancellation of the subscription, see {Unsubscribe}.
+  * **Must** support a way to send data to the subscriber.
+  * **May** include an initial response associated with executing the selection


Is the initial response supposed to be distinguishable from a regular response sent immediately after the subscription is started, from the point of view of the client?

Yes, I think the initial response should be distinguishable from a publish.

stubailo · 2017-05-08T07:14:38Z

spec/Section 6 -- Execution.md

+
+  * **Must** support observation of the associated publish stream (for example,
+  via iteration, callbacks, or reactive semantics).
+  * **Must** support cancellation of the subscription, see {Unsubscribe}.


I think using the word "cancel" here comes out of nowhere - maybe "must support unsubscribing" or "must have a way to stop the subscription"?

or "cancellation of the observation"

changed to "cancellation of the observation"

stubailo · 2017-05-08T07:17:53Z

spec/Section 6 -- Execution.md

+  * **Must** support observation of the associated publish stream (for example,
+  via iteration, callbacks, or reactive semantics).
+  * **Must** support cancellation of the subscription, see {Unsubscribe}.
+  * **Must** support a way to send data to the subscriber.


Is this referring to the event stream? I think this point could be more specific, since it's not clear what "data" is, or which entity is the "subscriber".

The event stream is an abstract concept for the client. All the client sees is a stream of 0-n "publishes", each of which contains a "payload". I feel the use of subscriber is fairly clear here. Opting to keep this as-is unless you feel very strongly otherwise.

OK I completely misinterpreted this. So "subscriber" refers to the client consuming GraphQL, and "send data" means "send GraphQL results".

stubailo · 2017-05-08T07:19:27Z

spec/Section 6 -- Execution.md

+      *normally* (allowing parallelization).
+    * Let {errors} be any *field errors* produced while executing the
+      selection set.
+    * If {errors} is not empty, optionally, Unsubscribe() the subscription.


Is this basically just saying the subscription can be terminated at any time by the backend? Or is there something special about terminating in response to a particular kind of error?

Yes, from the client's perspective, the subscription could be terminated at any point by the server. Nothing specific here about what sorts of errors result in termination.

That seems to make this bullet point unnecessary, since it refers to a thing that can happen at this time or any other time, or may not happen at all.

Agree w/ @stubailo - since this could also optionally unsubscribe the subscription if {errors} is empty, this line could be confusing.

I agree this bullet could just be removed here, but somewhere above mention that a subscription's stream of events may end at any moment, perhaps but not always in response to an error on the server

stubailo · 2017-05-08T07:24:04Z

spec/Section 6 -- Execution.md

+
+Rather than mixing stateless (queries and mutations) and stateful
+(subscriptions), we recommend keeping the GraphQL server stateless and
+delegating all state persistence to these sub-systems.


This almost sounds like it's recommending two schemas. I think what we are trying to say is that one thing should have all of the resolvers (for queries, mutations, and subscriptions), and another thing should have all of the state stuff. Perhaps:

Rather than mixing stateless and stateful logic in the same infrastructure, we recommend keeping your GraphQL execution system stageless, and delegating handling state about event streams and client connections to some other sub-system. In this design, the pub/sub system and client connection gateway call into the stateless execution layer when necessary to resolve GraphQL results. Production concerns like throttling and routing should be handled at the gateway layer to keep that complexity out of your API definition.

Agreed. I've reworded this section to try and highlight the main message better.

Rather than mixing stateless (queries and mutations) and stateful (subscriptions)

I think this still sounds like the above issue, because it is implying that everything about subscriptions should be separate. When in fact, I think subscription resolvers should probably be defined next to the regular schema, and just the event management and gateway needs to be separate.

stubailo · 2017-05-08T07:26:19Z

spec/Section 6 -- Execution.md

@@ -143,6 +147,96 @@ ExecuteMutation(mutation, schema, variableValues, initialValue):
    selection set.
  * Return an unordered map containing {data} and {errors}.

+### Subscription


Should we include some example responses here, and perhaps an example of a lifecycle for a simple subscription?

Good idea, I've added an example after the unsubscribe section.

wincent · 2017-05-09T18:47:54Z

Do you think it would be helpful to include a diagram? I think the RFC content was very helpful to give people a sense of what the text was trying to say, and I don't know if people can really construct an image of what is going on from this spec. Perhaps that's the job of the documentation, though.

@stubailo: I think you're right in what you say in that last sentence (emphasis added).

leebyron · 2017-05-11T22:56:24Z

spec/Section 6 -- Execution.md

+### Subscription
+
+If the operation is a subscription, the result of the operation is the creation
+of a persistent object on the server that exists for the lifetime of the


"object" is a strange word to use in this context, IMHO. Perhaps the creation of a persistent connection?

leebyron · 2017-05-11T23:04:54Z

spec/Section 6 -- Execution.md

+  * **Must** support cancellation of the observation, see {Unsubscribe}.
+  * **Must** support a way to send data to the subscriber.
+  * **May** include an initial response associated with executing the selection
+  set defined on the subscription operation.


I find this list of 4 things confusing and am having trouble understanding what exactly it is that it's describing. Specifically, these things seem to be describing an implementation detail that a client of GraphQL doesn't necessarily interact with or care about. For example, as a client of your GraphQL server, how would I test that you're adhering to these? For point 1, how would I know that the payloads I'm receiving are the associated to a "publish stream"? For point 2, why should I care? I can always just disconnect from the network. For point 3, this is obvious - nowhere else in the spec do we state a requirement that GraphQL must be used along with a way to send data to the requester. Point 4 is out of place as a "may" - it likely belongs elsewhere, probably as part of the "publish" section below.

What about writing lines 157-165 as:

The result of a subscription operation is a sequence of events over time, each event containing the result of executing the provided GraphQL selection on the data associated with that event.

I'm fine with collapsing the bullet points into the more concise form you have, but I think we'll lose point 4 in the process. I'll address this in your comment further down about initial responses.

leebyron · 2017-05-11T23:06:27Z

spec/Section 6 -- Execution.md

+data that tells us *who* logged on or off. Without this event data, the
+subscription cannot be evaluated. However, it is possible to define
+subscriptions that can be evaluated without event data (for example, the current
+time on the server, synthetically triggered every second). Subscriptions that do


A lot of this paragraph is describing implementation details, and IMHO is muddying the term "event data" used in context elsewhere. The example about an event triggered every second does contain event data - that event data is the current time on the server.

Some subscriptions use the event only as a trigger. These kinds of subscriptions can be evaluated whenever, and are capable of returning an initial value.

The "server time" subscription could be modeled either way. For example, if subscription's resolver calls Date.now() rather than expecting the time to come from the event data.

I can reword this section if you still think it's confusing, but I think it's important to call out this sub-class of subscription because this is why we bother broaching the topic of initial result further down.

leebyron · 2017-05-11T23:13:23Z

spec/Section 6 -- Execution.md

+  * Let {selectionSet} be the top level Selection Set in {subscription}.
+  * Let {rootField} be the first top level field in {selectionSet}.
+  * Let {eventStream} be the result of running {MapSubscriptionEvents(rootField, variableValues)}.
+  * Optionally: let {data} be the result of running {ExecuteRequest(schema, document, operationName, variableValues, initialValue)}, and return {data} on the subscription object.


I find this very confusing and probably not what we want. If I understand this correctly, sometimes this algorithm results in a value {data} which is returned, sometimes it does not - in which case it returns nothing?

Also, this algorithm does not describe what should be done with {eventStream} once created - I would have expected {eventStream} to be returned.

As a concrete example - what should I expect to see differently between a subscription which has an initial response vs one that does not, say in terms of websocket payloads? Do I actually get a {} or {data: null} payload as an "ACK" for subscriptions without an initial value? Or is "ACK" a detail that belongs to the transport mechanism?

Perhaps another way to frame this is that "initial value" could be a first published event which simply occurs instantaneously after the creation of a subscription?

leebyron · 2017-05-11T23:21:22Z

spec/Section 6 -- Execution.md

+
+MapSubscriptionEvents(rootField, variableValues):
+
+  * *Application-specific logic to map from root field/variables to events*


This should probably be filled in. In particular, nothing is referencing the algorithm in Publish yet, which should probably be referenced here.

Or perhaps there's something missing? By name, I would expect this algorithm to map one set of events to another. Maybe this one should be renamed CreateSubscriptionEvents or something along those lines which should actually just be a delegation to provided logic, and then separately include MapSubscriptionEvents which calls into the Publish algo.

Also - this section on mapping event streams is a great opportunity to mention that while described as an inline algorithm, this step is often implemented across services, with the original event stream belonging to a subscription management service, and the execution happening on an API middleware service.

Yeah, you're right, "map" is misleading here. Changed to "CreateSubscriptionEvents" and hooked up the pseudocode from above and below.

leebyron · 2017-05-11T23:21:41Z

spec/Section 6 -- Execution.md

+event carries an optional payload, which we combine with the arguments from
+Subscribe() to resolve the selection set.
+
+  * For each {event} and {eventData} on {eventStream}:


This algo needs a name and arguments

Also, the For each {event} ... line likely belongs in the MapSubscriptionEvents such that Publish can be written in a way that is independent to each publish action

Is there value to describing {event} and {eventData} separately? For example, {event} is never referenced below - what would be its value?

leebyron · 2017-05-11T23:25:46Z

spec/Section 6 -- Execution.md

+      *normally* (allowing parallelization).
+    * Let {errors} be any *field errors* produced while executing the
+      selection set.
+    * If {errors} is not empty, optionally, Unsubscribe() the subscription.


Agree w/ @stubailo - since this could also optionally unsubscribe the subscription if {errors} is empty, this line could be confusing.

I agree this bullet could just be removed here, but somewhere above mention that a subscription's stream of events may end at any moment, perhaps but not always in response to an error on the server

leebyron · 2017-05-11T23:26:51Z

spec/Section 6 -- Execution.md

+Unsubscribe()
+
+  * Terminate {publishStream} (For example, cancel iteration, detach mapping function)
+  * Terminate and clean up {eventStream}


This whole section seems like implementation detail and not really relevant to what is or isn't GraphQL, could it be collapsed into a footnote in the top section on subscriptions?

robzhu

Reworked this section.

robzhu · 2017-05-12T16:21:38Z

spec/Section 6 -- Execution.md

+
+MapSubscriptionEvents(rootField, variableValues):
+
+  * *Application-specific logic to map from root field/variables to events*


Yeah, you're right, "map" is misleading here. Changed to "CreateSubscriptionEvents" and hooked up the pseudocode from above and below.

leebyron

This is looking awesome. More fine-grained feedback this time

leebyron · 2017-05-13T02:06:26Z

spec/Section 6 -- Execution.md

@@ -103,8 +103,9 @@ Note: This algorithm is very similar to {CoerceArgumentValues()}.
 ## Executing Operations

 The type system, as described in the “Type System” section of the spec, must
-provide a query root object type. If mutations are supported, it must also
-provide a mutation root object type.
+provide a query root object type. If mutations or subscriptions are supported, it must also provide a mutation and subscription root object type, respectively.


nit: line break to maintain roughly 80c?

leebyron · 2017-05-13T02:24:35Z

spec/Section 6 -- Execution.md

+future events.
+
+If the operation is a subscription, the result is an event stream called the
+"Publish Stream" where each event in the stream is called a "Publish Payload".


Flip the order of these two paragraphs.

Also for the para on event streams, some additional detail is necessary around event stream completion. Maybe also some examples for color. Perhaps:

An event stream represents a sequence of discrete events over time which can be observed. As an example, a "Pub-Sub" system may produce an event stream when "subscribing to a topic", with an event occurring on that event stream for each "publish" to that topic. Event streams may produce an infinite sequence of events or may complete at any point. Event streams may complete in response to an error or simply because no more events will occur. An observer may at any point decide to stop observing an event stream, after which it must receive no more events from that event stream.

Note: If an event stream's observer has stopped observing, that may be a good opportunity to clean up any associated resources such as closing any connections which are no longer necessary.

For the other paragraph, I'm not sure the events in the stream need a name so much as they need a brief description:

If the operation is a subscription, the result is an event stream called the "Response Stream" where each event in the event stream is the result of executing the operation for each new event on an underlying "Source Stream".

leebyron · 2017-05-13T02:29:47Z

spec/Section 6 -- Execution.md

+future events.
+
+If the operation is a subscription, the result is an event stream called the
+"Publish Stream" where each event in the stream is called a "Publish Payload".


I fear "Publish Stream" may be too ambiguous - it could be too easy to confuse it with the underlying stream. How about: "Source Stream" and "Response Stream"?

leebyron · 2017-05-13T02:37:03Z

spec/Section 6 -- Execution.md

+
+CreateUnderlyingEventStream(rootField, variableValues):
+
+  * *Application-specific logic to map from root field/variables to events*


As a reflection of the feedback on graphql/graphql-js#846 - let's break out the top portion of Subscribe and name this second algorithm "Resolve" since it's resolving from a field.

CreateSourceEventStream(schema, subscription, operationName, variableValues, initialValue)

ResolveFieldEventStream(subscriptionType, rootValue, fieldName, argumentValues) <- should mirror ResolveFieldValue - look there for an example of getting these values

Once the Subscribe() step is only these two steps, I think you could make a reference to your bottom most section by adding a Note: to clarify that in many real services, this Subscribe algorithm is run on a separate service than the execute step, and to refer to the section below on Considerations when supporting subscriptions

leebyron · 2017-05-13T02:39:24Z

spec/Section 6 -- Execution.md

+Each event in the underlying event stream triggers execution of the subscription
+selection set.
+
+MapEventToPayload(eventStream):


Missing the arguments referred to below

Perhaps MapSourceToResponseEventStream?

leebyron · 2017-05-13T02:52:13Z

spec/Section 6 -- Execution.md

+
+#### Recommendations and Considerations for Supporting Subscriptions
+
+Supporting subscriptions is a large change for any GraphQL server. Query and


"significant" instead of "large"?

leebyron · 2017-05-13T02:56:00Z

spec/Section 6 -- Execution.md

+}
+```
+
+#### Recommendations and Considerations for Supporting Subscriptions


This is a long section title and it's specifically about scale. So perhaps just Supporting Subscriptions at Scale?

leebyron · 2017-05-13T02:59:11Z

spec/Section 6 -- Execution.md

+  * Subscription document
+  * Variables
+  * Execution context (for example, current logged-in user, locale, etc.)
+  * Event stream resulting from Subscribe step (above)


I find this list of things a bit confusing and overly detailed for the point you're trying to make. Perhaps just:

Subscriptions, by contrast, are long-lived and stateful and require maintaining the GraphQL document, variables, and other context over the lifetime of the request.

leebyron · 2017-05-13T03:00:52Z

spec/Section 6 -- Execution.md

+  * Execution context (for example, current logged-in user, locale, etc.)
+  * Event stream resulting from Subscribe step (above)
+
+We recommend thinking about the behavior of your system when this state is lost


Spec shouldn't speak in the third person. Instead:

Consider the behavior of your system when state is lost due to the failure of a single machine in a service. Durability and availability may be improved by having a separate dedicated service for managing this state.

leebyron · 2017-05-13T03:01:47Z

spec/Section 6 -- Execution.md

+recommend keeping the GraphQL server stateless and delegating all state
+persistence to sub-systems that are designed for stateful scaling. Note that
+subscription types are still defined in the original schema along with queries
+and mutations.


This note is a bit out of place. You can define this stuff wherever the heck you want, that's implementation detail. Also, this paragraph is really just repeating the point made in the previous one. I think you can probably just remove it entirely if not move a few of the more fine-pointed words into the prior paragraph

robzhu

Thanks for the thorough review! I think I addressed all of your comments.

robzhu · 2017-05-13T05:22:39Z

spec/Section 6 -- Execution.md

+      *normally* (allowing parallelization).
+    * Let {errors} be any *field errors* produced while executing the
+      selection set.
+    * Yield an unordered map containing {publishPayload} and, optionally,


It's pretty close after the accumulated edits.

robzhu · 2017-05-13T05:29:34Z

spec/Section 6 -- Execution.md

+      selection set.
+    * Yield an unordered map containing {publishPayload} and, optionally,
+      {errors} on {publishStream}.
+  * At any time while the publish stream is active, the client or server may


I think this is clearly from the "for each" iteration of {sourceStream}.

leebyron

Getting excited! Most feedback here is within the algorithms

leebyron · 2017-05-13T18:54:39Z

spec/Section 6 -- Execution.md

+#### Subscribe
+
+Executing a subscription creates a persistent function on the server that
+maps an underlying event stream to the Publish Stream. The logic to create the


maps an underlying Source Stream to a returned Response Stream

leebyron · 2017-05-13T18:54:56Z

spec/Section 6 -- Execution.md

+
+Executing a subscription creates a persistent function on the server that
+maps an underlying event stream to the Publish Stream. The logic to create the
+underlying event stream is application-specific and takes the root field and


to create the Source Stream is application-specific

leebyron · 2017-05-13T18:57:09Z

spec/Section 6 -- Execution.md

+underlying event stream is application-specific and takes the root field and
+query variables as inputs.
+
+Subscribe(schema, subscription, operationName, variableValues, initialValue):


operationName isn't used here. Other related algos like ExecuteQuery look like:

ExecuteQuery(query, schema, variableValues, initialValue):

So perhaps:

ExecuteSubscription(subscription, schema, variableValues, initialValue):

leebyron · 2017-05-13T18:58:37Z

spec/Section 6 -- Execution.md

+  * Let {subscriptionType} be the root Subscription type in {schema}.
+  * Assert: {subscriptionType} is an Object type.
+  * Let {selectionSet} be the top level Selection Set in {subscription}.
+  * Let {sourceStream} be the result of running {CreateSourceEventStream(schema, subscription, selectionSet, variableValues, initialValue)}.


The 3 steps above this one should probably be a step in CreateSourceEventStream. That algo's arguments should mirror that of this one.

leebyron · 2017-05-13T18:59:16Z

spec/Section 6 -- Execution.md

+  * Let {selectionSet} be the top level Selection Set in {subscription}.
+  * Let {sourceStream} be the result of running {CreateSourceEventStream(schema, subscription, selectionSet, variableValues, initialValue)}.
+  * Let {responseStream} be the result of running
+    {MapSourceToResponseEvent(sourceStream)}


one line for this should be fine

leebyron · 2017-05-13T19:01:30Z

spec/Section 6 -- Execution.md

+      {errors}.
+    * Yield an event containing {response}.
+
+Note: in large scale subscription systems, ExecuteSelectionSet and Subscribe


nit: {} wrapped around algo names so they link in the published html

leebyron · 2017-05-13T19:02:21Z

spec/Section 6 -- Execution.md

+      *normally* (allowing parallelization).
+    * Let {errors} be any *field errors* produced while executing the
+      selection set.
+    * Let {response} be an unordered map containing {publishPayload} and, optionally,


this is a bit ambiguous since we expect the unordered map to use the key data but this implies using a key publishPayload - maybe just rename the variable

leebyron · 2017-05-13T19:04:09Z

spec/Section 6 -- Execution.md

+Each event in the underlying event stream triggers execution of the subscription
+selection set.
+
+MapSourceToResponseEvent(sourceStream, subscriptionType, selectionSet, variableValues):


This should probably operate on the whole subscription instead of selectionSet and schema instead of subscriptionType - and pass those through to another algorithm which is a mirror of ExecuteQuery - that becomes the seam for passing the command to another system. Maybe call it ExecuteSubscriptionEvent?

leebyron · 2017-05-13T19:04:33Z

spec/Section 6 -- Execution.md

+
+#### Unsubscribe
+
+Unsubscribe cancels the Publish Stream. This is also a good opportunity for the


cancels the Response Stream?

robzhu · 2017-05-13T19:36:11Z

I took another pass and made changes per your suggestions. Reading everything top-to-bottom, it feels a lot more dense/verbose. We also have some duplication, for example with ExecuteSubscriptionEvent. Do you feel this restructuring buys us consistency/clarity?

leebyron

There are some other sections which need some updates:

http://facebook.github.io/graphql/#sec-Language.Operations

http://facebook.github.io/graphql/#sec-Initial-types

http://facebook.github.io/graphql/#sec-Schema-Introspection

http://facebook.github.io/graphql/#sec-Appendix-Grammar-Summary.Query-Document

leebyron · 2017-05-14T22:17:54Z

spec/Section 6 -- Execution.md

@@ -143,6 +143,84 @@ ExecuteMutation(mutation, schema, variableValues, initialValue):
    selection set.
  * Return an unordered map containing {data} and {errors}.


Also noticed that a step should be added to {ExecuteRequest} above to handle subscription requests.

leebyron · 2017-05-14T22:22:30Z

spec/Section 6 -- Execution.md

+Source stream is application-specific and takes the root field and query
+variables as inputs.
+
+Subscribe(subscription, schema, variableValues, initialValue):


Should this be called ExecuteSubscription to mirror the other algos called by ExecuteRequest?

I'm concerned that "ExecuteSubscription" and "ExecuteSubscriptionEvent" will be confused with one another. The name seems to imply that the caller can expect a response containing data.

leebyron · 2017-05-14T22:23:49Z

spec/Section 6 -- Execution.md

+ResolveFieldEventStream(subscriptionType, rootValue, fieldName, argumentValues):
+  * Let {resolver} be the internal function provided by {subscriptionType} for
+    determining the resolved value of a field named {fieldName}.
+  * Return the result of calling {resolver}, providing {rootValue} and {argumentValues}.


It would be nice to add a Note: below this algorithm mentioning its intentional similarity to {ResolveFieldValue}

leebyron · 2017-05-14T22:24:12Z

spec/Section 6 -- Execution.md

+    determining the resolved value of a field named {fieldName}.
+  * Return the result of calling {resolver}, providing {rootValue} and {argumentValues}.
+
+#### Publish Stream


"Response Stream"?

leebyron · 2017-05-14T22:25:23Z

spec/Section 6 -- Execution.md

+    *normally* (allowing parallelization).
+  * Let {errors} be any *field errors* produced while executing the
+    selection set.
+  * Return an unordered map containing {data} and {errors}.


Similarly, it would be nice to have a Note: which mentions the intentional similarity to {ExecuteQuery}

leebyron · 2017-05-14T22:28:09Z

spec/Section 6 -- Execution.md

+
+Unsubscribe()
+
+  * Cancel {responseStream}


Above we used the term "stop observing" - we should probably be consistent here.

Since the server is the producer of the response stream, if we use "stop observing", we probably give the impression that we are describing client behavior, whereas the previous sections clearly describe server behavior.

leebyron · 2017-05-14T22:30:07Z

spec/Section 6 -- Execution.md

+the subscription. Here are some example cases in which to Unsubscribe: client
+no longer wishes to receive payloads for a subscription; the source event stream
+produced an error or naturally ended; the server encountered an error during
+ExecuteSubscriptionEvent.


I agree with you that this is redundant with the note above. I think you should merge the two together, either relying on the Note above and leaving this section with a much simpler explanation of the algorithm below, or removing the Note above and leaving those details here.

Removed the note in the "### Subscription" section and left this paragraph in place. I think this reads more clearly after moving the example.

leebyron · 2017-05-14T22:31:36Z

spec/Section 6 -- Execution.md

+
+  * Cancel {responseStream}
+
+#### Example


Elsewhere we use **Example** to avoid creating sections in the TOC for examples. Also, what do you think about moving the Example directly above the algorithms as the last part of the top level "Subscription" section?

Great suggestion!

leebyron · 2017-05-14T22:32:46Z

spec/Section 6 -- Execution.md

+}
+```
+
+#### Supporting Subscriptions at Scale


You might consider also moving this section above the algorithms and making this a **Title** instead so it's more directly linked to the discussion of subscriptions.

leebyron · 2017-05-14T22:37:52Z

We also have some duplication, for example with ExecuteSubscriptionEvent. Do you feel this restructuring buys us consistency/clarity?

Yes, I think that it helps to illustrate the seam between creating a subscription and executing each event, which will help those building this across two services.

robzhu

Back over to you, @leebyron

robzhu · 2017-05-15T14:06:49Z

spec/Section 6 -- Execution.md

+Source stream is application-specific and takes the root field and query
+variables as inputs.
+
+Subscribe(subscription, schema, variableValues, initialValue):


I'm concerned that "ExecuteSubscription" and "ExecuteSubscriptionEvent" will be confused with one another. The name seems to imply that the caller can expect a response containing data.

robzhu · 2017-05-15T15:03:15Z

spec/Section 6 -- Execution.md

+
+Unsubscribe()
+
+  * Cancel {responseStream}


Since the server is the producer of the response stream, if we use "stop observing", we probably give the impression that we are describing client behavior, whereas the previous sections clearly describe server behavior.

robzhu · 2017-05-15T15:04:54Z

spec/Section 6 -- Execution.md

+the subscription. Here are some example cases in which to Unsubscribe: client
+no longer wishes to receive payloads for a subscription; the source event stream
+produced an error or naturally ended; the server encountered an error during
+ExecuteSubscriptionEvent.


Removed the note in the "### Subscription" section and left this paragraph in place. I think this reads more clearly after moving the example.

robzhu · 2017-05-15T15:13:34Z

spec/Section 6 -- Execution.md

+
+  * Cancel {responseStream}
+
+#### Example


Great suggestion!

leebyron · 2017-05-17T16:23:15Z

spec/Section 6 -- Execution.md

+      * If {operation} is a subscription operation:
+        * If {operation} contains more than one root field, produce a query error.
+        * Return {operation}.
+      * Otherwise return {operation}.


These new steps aren't necessary. The validation rule will check for this, and the CreateSourceEventStream already does the right thing of using the first (only) field in the set

leebyron · 2017-05-17T16:24:21Z

spec/Section 2 -- Language.md


  * query - a read-only fetch.
  * mutation - a write followed by a fetch.
+  * subscription - a long-lived request that returns data whenever a domain
+    event triggers.


to recycle terms, how about: "a long-lived request that fetches data in response to source events."

leebyron · 2017-05-17T16:28:33Z

This looks awesome. Imminently mergeable.

…tion

leebyron · 2017-05-17T22:16:52Z

This turned out great.

josephsavona · 2017-05-17T22:39:29Z

Nice job with this, @robzhu!!!

* Subscription validation rule: single root field * typo in explanatory text * Updated wording to match Execution section * Execution section for subscriptions * Remove leftover section on Subscription functions * Incorporate feedback from first review * Line feed after subscription sections * Fix spec syntax * Address wincent's feedback * Modify wording from Laney's feedback * Address Sashko's feedback * Fix inline code snippets * Address Lee's feedback * Address second round of feedback w/ Lee * Address third round of feedback from Lee * Add subscriptions to other sections * Address more feedback on Execution section * Remove extra single root field check and reword subscriptions description * and/or * Minor editing updates * Mini-header for event streams

Robert Zhu added 6 commits May 2, 2017 14:28

Subscription validation rule: single root field

4bba743

typo in explanatory text

7b4fb56

Updated wording to match Execution section

328644b

Execution section for subscriptions

c4aeaf9

Remove leftover section on Subscription functions

f38092b

Incorporate feedback from first review

ca8af4d

facebook-github-bot added the CLA Signed label May 2, 2017

Robert Zhu added 2 commits May 2, 2017 17:34

Line feed after subscription sections

3933a5a

Fix spec syntax

d3220d6

wincent reviewed May 2, 2017

View reviewed changes

Address wincent's feedback

e61963c

DxCx reviewed May 3, 2017

View reviewed changes

laneyk reviewed May 3, 2017

View reviewed changes

Modify wording from Laney's feedback

6d9dc8d

wincent mentioned this pull request May 5, 2017

How to do live-reload graphql/express-graphql#296

Closed

stubailo reviewed May 8, 2017

View reviewed changes

Urigo mentioned this pull request May 9, 2017

Subscriptions support graphql/graphql-js#846

Merged

Fix inline code snippets

d3011a5

leebyron reviewed May 11, 2017

View reviewed changes

robzhu commented May 12, 2017

View reviewed changes

Address Lee's feedback

23bd790

syrusakbary mentioned this pull request May 13, 2017

Graphene real-time subscriptions and Apollo client graphql integration graphql-python/graphene#430

Closed

leebyron reviewed May 13, 2017

View reviewed changes

Address second round of feedback w/ Lee

f7581ef

robzhu commented May 13, 2017

View reviewed changes

leebyron reviewed May 13, 2017

View reviewed changes

Address third round of feedback from Lee

f18c569

leebyron reviewed May 14, 2017

View reviewed changes

Robert Zhu added 2 commits May 15, 2017 01:25

Add subscriptions to other sections

a0aac7f

Address more feedback on Execution section

35c6059

robzhu commented May 15, 2017

View reviewed changes

leebyron reviewed May 17, 2017

View reviewed changes

Robert Zhu and others added 4 commits May 17, 2017 13:09

Remove extra single root field check and reword subscriptions descrip…

ca2a02e

…tion

and/or

36359fd

Minor editing updates

47404cb

Mini-header for event streams

6425fed

leebyron merged commit d7e3f06 into graphql:master May 17, 2017

jorroll mentioned this pull request Dec 27, 2019

Support batched queries (fix #1812) hasura/graphql-engine#3490

Merged

27 tasks

benjie mentioned this pull request Dec 14, 2023

Fix formatting and typo in MapSourceToResponseEvent #1070

Closed

		@@ -143,6 +143,84 @@ ExecuteMutation(mutation, schema, variableValues, initialValue):
		selection set.
		* Return an unordered map containing {data} and {errors}.


		Publish(subscription, schema, variableValues, payload, publishStream)

		* For each {eventStream} as {event} and {payload}:


		Explanatory Text

		Subscription operations must have exactly one root field.


		MapSubscriptionEvents(rootField, variableValues):

		* Application-specific logic to map from root field/variables to events


		CreateUnderlyingEventStream(rootField, variableValues):

		* Application-specific logic to map from root field/variables to events


		#### Recommendations and Considerations for Supporting Subscriptions

		Supporting subscriptions is a large change for any GraphQL server. Query and


		#### Unsubscribe

		Unsubscribe cancels the Publish Stream. This is also a good opportunity for the

Spec: Formal Subscriptions Definition #305

Spec: Formal Subscriptions Definition #305

Conversation

robzhu commented May 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OlegIlyenko May 9, 2017 • edited Loading

Choose a reason for hiding this comment

OlegIlyenko May 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robzhu May 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OlegIlyenko May 12, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robzhu May 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robzhu May 12, 2017 • edited Loading

Choose a reason for hiding this comment

stubailo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wincent commented May 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OlegIlyenko May 9, 2017 •

edited

Loading

OlegIlyenko May 9, 2017 •

edited

Loading

robzhu May 11, 2017 •

edited

Loading

OlegIlyenko May 12, 2017 •

edited

Loading

robzhu May 3, 2017 •

edited

Loading

robzhu May 12, 2017 •

edited

Loading