Not all mutations delivered to subscribed client

[beepsoft]

I have a situation where not all mutation results are delivered to a subscription client.

I have an apollo SubscriptionClient executing this subscription, which would give me the changed records after $startDate (which is the current date when the subscription is executed)

subscription inProgress($startDate: timestamp) {
    deliveredInstructions(
        order_by: {updatedAt:desc}
        where:{
            updatedAt:{
                _gt: $startDate
            }
        }
    )
    {
        __typename
        id
        parameters
        updatedAt		
    }
}

I have a database trigger, which makes sure the updated_at field (which is renamed to updatedAt in the schema) is set to the current date whenever the record is updated.

Now if I execute the following mutation periodically (via bash for i in {1..5}; do; curl ...; done) in my client I will only receive the result of the fifth mutation (in my script I also set the values 1, 2, 3, 4, 5 in parameters):

mutation setParam {
  updateDeliveredInstructions(
        where:{id:{_eq:61}}
  	_set: {
            parameters: "..."
         }
  ) {
		returning {
      id
      parameters
      updatedAt
    }
  }
}

This is the periodic execution result. You can see updatedAt is properly set to current date by the trigger:

> ./substest.sh 
Caling 1
{"data":{"updateDeliveredInstructions":{"returning" : [{"id":"61","parameters":"1","updatedAt":"2019-12-11T09:37:54.411028"}]}}}
Caling 2
{"data":{"updateDeliveredInstructions":{"returning" : [{"id":"61","parameters":"2","updatedAt":"2019-12-11T09:37:54.481134"}]}}}
Caling 3
{"data":{"updateDeliveredInstructions":{"returning" : [{"id":"61","parameters":"3","updatedAt":"2019-12-11T09:37:54.522796"}]}}}
Caling 4
{"data":{"updateDeliveredInstructions":{"returning" : [{"id":"61","parameters":"4","updatedAt":"2019-12-11T09:37:54.562296"}]}}}
Caling 5
{"data":{"updateDeliveredInstructions":{"returning" : [{"id":"61","parameters":"5","updatedAt":"2019-12-11T09:37:54.600819"}]}}}

In Chrome's websocket inspector I see the following for the above executed queries:

{"type":"connection_init","payload":{"headers":{"X-Hasura-Admin-Secret":"XXX"}}}	82	
10:37:45.336
{"id":"1","type":"start","payload":{"query":"subscription inProgress($startDate: timestamp) {\n deliveredInstructions(order_by: {updatedAt: desc}, where: {_and: [{vendorData: {_contains: {vendorId: \"sztaki-uccps\"}}}, {updatedAt: {_gt: $startDate}}, {statusValue: {_eq: DELIVERED}}]}) {\n __typename\n id\n parameters\n updatedAt\n }\n}\n","variables":{"startDate":"2019-12-11T09:37:45Z"}}}	405	
10:37:45.336
{"type":"ka"}	13	
10:37:45.347
{"type":"connection_ack"}	25	
10:37:45.408
{"type":"ka"}	13	
10:37:45.416
{"type":"data","id":"1","payload":{"data":{"deliveredInstructions":[]}}}	72	
10:37:45.416
{"type":"ka"}	13	
10:37:50.023
{"type":"ka"}	13	
10:37:55.021
{"type":"data","id":"1","payload":{"data":{"deliveredInstructions":[{"__typename":"delivered_instruction","id":"61","parameters":"5","updatedAt":"2019-12-11T09:37:54.600819"}]}}}	178	
10:37:55.411
{"type":"ka"}

As you can see I only received the result of the last mutation. If I put a sleep 1 in the execution loop then all mutation result gets delivered, though.

So my questions are:

• Is this the expected behaviour or am I doing something wrong?
• Are there subscription delivery guarantees?
• Is there some logic throttling in subscription result generation so that if a record changes "too much" in a given time range then it would not be delivered to a subscription?

[yuvalgut]

Subscriptions update every 1 second. You can find it in the docs

[beepsoft]

@yuvalgut Really? Can you link it, please? I've read through the documentation many times and searched github issues, etc. but haven't find any reference to such a restriction.

[yuvalgut]

https://github.com/hasura/graphql-engine/blob/master/architecture/live-queries.md

[beepsoft]

Thanks! I know this one as well, but this only describes this experiment with 1 million active GraphQL subscriptions where they happen to update the database every second. But it is not a proof and not an official documentation on how the subscription mechanism works or intended to work and whether there's such a 1 second update rule in it.

[rikinsk]

@beepsoft The implementation details are mentioned at https://github.com/hasura/graphql-engine/blob/master/architecture/live-queries.md#implementing-graphql-live-queries.

We'll add a table of contents to the document to make this more easy to discover

[beepsoft]

@rikinsk Thanks! I've read the architecture but I still can't find there what are the actual constraints of subscriptions and how frequently updates actually get delivered? An update every second? Every 500ms? Some other metrics? It would be great to have these defined in the relevant Hasura documentation (eg. at https://docs.hasura.io/1.0/graphql/manual/subscriptions/index.html)

And it would be also helpful to provide tips there how to handle situations when the client wants to receive every update. Eg. how frequently should the client execute an additional query to get those updates that the subscription mechanism may missed. Or should the client do a query every time it receives some data via the subscription to make sure it gets all updates: this way the subscription would only serve as a "ping" to inform the client that some data changed and need to fetch actual the changes manually.

Thanks for your efforts!

[rikinsk]

@beepsoft As you would have figured out by reading the architecture doc, Hasura subscriptions are actually live-queries, hence they do not guarantee sending an update for every change.

By default updates get delivered every 1sec. This interval can be configured via the HASURA_GRAPHQL_LIVE_QUERIES_MULTIPLEXED_REFETCH_INTERVAL env var or the --live-queries-multiplexed-refetch-interval flag. You can check the server config docs for details.

Handling of the use-case as described by you is currently under discussion. We'll only be able to share more details around that when we have something concrete.

cc: @tirumaraiselvan

[beepsoft]

@rikinsk Thanks for the details, it is really helpful!

Maybe it was just my wrong interpretation of the 3factory.app theory, but I thought this is kind of essential that the same way event triggers guarantee that for ever change on a table a URL will be called for servers to handle clients should also have a way to get informed of all changes relevant for them. I understand that the current subscription functionality is not (completely) designed for that.

It would be great to have an idiomatic Hasura solution for this. Until then I can work it around with interpreting subscription results as just signals of changes, which then would trigger an additional query to get the actual updates.

[rikinsk]

I can work it around with interpreting subscription results as just signals of changes, which then would trigger an additional query to get the actual updates.

Just out of curiosity, can you share your use case where you really need this be done. For most typical client applications, it is usually enough to just have the latest response for a query (if we have the right query for the job of course).

[beepsoft]

Well, my current use case doesn't actually need it I just wanted to know what are the limits of subscriptions. In my current application work instructions will be delivered to workers and these will be delivered less frequently than 1 second, of course.

However I can imagine a chat application where people chat in a group and each client wants to get the latest message. In case there are many members many messages could be created in the same second. How can I guarantee that everyone will get all new messages via subscriptions that were created in that same second and I don't want to store the delivery state of the message per client in the DB?

My idea is that I would use a subscription with conditions to get the latest created 1 message and I would get all the messages created in that one second one-by-one this way. This would not work with the current subscription implementation, I guess. However, if I stored delivery state of the messages, then I could subscribe for all messages that have not been delivered to me, and in that case I would probably get all the messages in that 1 second with no problem.

[tirumaraiselvan]

@beepsoft Please see #2317

[marionschleifer]

@beepsoft I'm closing this issue. If you would like to add anything, feel free to re-open 🙂