feat: request/response support in both operation and message objects #594
Conversation
|
Kudos, SonarCloud Quality Gate passed!
|
spec/asyncapi.md
Outdated
| @@ -1018,6 +1039,7 @@ Field Name | Type | Description | |||
| <a name="messageObjectBindings"></a>bindings | [Message Bindings Object](#messageBindingsObject) \| [Reference Object](#referenceObject) | A map where the keys describe the name of the protocol and the values describe protocol-specific definitions for the message. | |||
| <a name="messageObjectExamples"></a>examples | [Map[`string`, `any`]] | An array of key/value pairs where keys MUST be either **headers** and/or **payload**. Values MUST contain examples that validate against the [headers](#messageObjectHeaders) or [payload](#messageObjectPayload) fields, respectively. Example MAY also have the **name** and **summary** additional keys to provide respectively a machine-friendly name and a short summary of what the example is about. | |||
| <a name="messageObjectTraits"></a>traits | [[Message Trait Object](#messageTraitObject) | [Reference Object](#referenceObject)] | A list of traits to apply to the message object. Traits MUST be merged into the message object using the [JSON Merge Patch](https://tools.ietf.org/html/rfc7386) algorithm in the same order they are defined here. The resulting object MUST be a valid [Message Object](#messageObject). | |||
| <a name="messageObjectResponse"></a>response | [[Message Object](#messageObject) | [Reference Object](#referenceObject)] | A OPTIONAL definition of the message that the other party can expect to receive as a result to publish operation using this Message Object or as a result to receiving this Message Object in subscription. oneOf is allowed here to specify multiple messages, however, **a message MUST be valid ony against on of the referenced message objects.** | |||
There was a problem hiding this comment.
What's the reasoning behind having response inside Message Object? Isn't it enough to have it at the operation level?
Also, how does it play with the response field on the operation? E.g.:
channels:
mychannel:
publish:
message:
response: ...
response: ...What does it mean? Does it mean the one at the operation level prevails over the one at the message level? Maybe both could be expected? If so, how does it work when you have two oneOfs on each response field?
There was a problem hiding this comment.
As mentioned in initial "Identification of potential concerns, challenges, and drawbacks:" number (2).
The reasons are 2
- Variability to support as wide spectrum of protocol binding as possible
- Backward compatibility with current
x-responseextension as documented by @derberg in blog websocket-part2
About the implementation, if both operation and message level response are filled in (even with oneOf) the parser/interpretation shall evaluate/validate/list all possible messages (merge of both response oneOf lists). So this design is intentional.
This also allows for operation level error responses and message level typed responses, eg. as DRY as possible
There was a problem hiding this comment.
don't worry about x-response, it is just concept, not even official extension.
I think having it on operation level is anyway not clear. In the end we talk about reply to a message and not the operation, so intuition automatically tells me to look for it inside the message. It also makes sense for oneOf use case, where you have multiple different messages and depending on the message, you can have different reply. This is why I did it this way in my blog post. In the world of WebSocket in cryptocurrency trading APIs, you have one channel, and multiple different messages, when you send message ping you get pong in reply, and when you send message subscriptionStatus, you set status message in response. So depending on the message, you get different reply.
messages:
ping:
summary: Ping server to determine whether connection is alive
description: Client can ping server to determine whether connection is alive, server responds with pong. This is an application level ping as opposed to default ping in websockets standard which is server initiated
payload:
$ref: '#/components/schemas/ping'
x-response:
$ref: '#/components/messages/pong'
Keep in mind that my blog post is based on real API, Kraken API, and Gemini in v2 follows the same pattern.
@jonaslagoni how does it work with NATS? what would be better there?
There was a problem hiding this comment.
@derberg each response could in theory be a response to a specific message. There are no limitations there (guess it comes down to implementation) 🙂 So the response operation keyword does not make much sense there either.
There was a problem hiding this comment.
@derberg understood, however what i had in mind, was allowing to put the general responses in channel (channel can report authentication, authorization, availability or different status itself, not corresponding to sent message) and message-specific responses linked with message, not channel.
@jonaslagoni i don't really have preference over the naming, so i'm open to suggestions
There was a problem hiding this comment.
@smarek I think what @jonaslagoni meant was that from his perspective and how it worked in NATS is that response is enough on message level and not on operation level. At least this is my assumption 😅
There was a problem hiding this comment.
Could be, but that would require linking all the operation-level responses to all the possible messages that can be processes in that operation/channel, which seemed not-DRY to me, which is also why i proposed operation response as deduplication approach.
There was a problem hiding this comment.
but when I specify that pong is a response to ping and that subscriptionStatus is response to subscription I do not validate any DRY principles, right?
message can be one or oneOf this response on operation level only confuses. If I have just one message, I'll specify it in this one message, but if I have oneOf then I will specify it there in every message. When would I do it on operation level?
There was a problem hiding this comment.
well for your consideration, which of these seems better to you? (please ignore invalid schema, this is just to illustrate the RFC intent)
just-message
channels:
basic:
message:
oneOf:
- $ref: '#/components/messages/ping'
- $ref: '#/components/messages/config'
- $ref: '#/components/messages/update'
components:
messages:
ping:
payload:
$ref: '#/components/schemas/ping'
response:
oneOf:
- $ref: '#/components/messages/pong'
- $ref: '#/components/messages/unauthorized'
- $ref: '#/components/messages/unauthenticated'
- $ref: '#/components/messages/service_restarting'
config:
payload:
$ref: '#/components/schemas/config'
response:
oneOf:
- $ref: '#/components/messages/config_result'
- $ref: '#/components/messages/unauthorized'
- $ref: '#/components/messages/unauthenticated'
- $ref: '#/components/messages/service_restarting'
update:
payload:
$ref: '#/components/schemas/update'
response:
oneOf:
- $ref: '#/components/messages/update_result'
- $ref: '#/components/messages/unauthorized'
- $ref: '#/components/messages/unauthenticated'
- $ref: '#/components/messages/service_restarting'channel and message
channels:
basic:
message:
oneOf:
- $ref: '#/components/messages/ping'
- $ref: '#/components/messages/config'
- $ref: '#/components/messages/update'
response:
oneOf:
- $ref: '#/components/messages/unauthorized'
- $ref: '#/components/messages/unauthenticated'
- $ref: '#/components/messages/service_restarting'
components:
messages:
ping:
payload:
$ref: '#/components/schemas/ping'
response:
$ref: '#/components/messages/pong'
config:
payload:
$ref: '#/components/schemas/config'
response:
$ref: '#/components/messages/config_result'
update:
payload:
$ref: '#/components/schemas/update'
response:
$ref: '#/components/messages/update_result'In my opinion the channel and message configuration is more clear about response messages belonging to channel operation status and also linking the specific request to one or multiple valid responses. It is not mandatory to define responses on channel or message, but many types/implementations of API are layered (and for good reasons) and individual layers can/may produce responses not relevant to request message but relevant to service/user/operational status/level
There was a problem hiding this comment.
While I do see your use-case @smarek, I would suggest focusing on what matters here, enabling request/reply, not improving a feature we don't have. Request/reply discussion is complex enough, that adding more into the discussion, won't benefit or focus on what matters 🙂
I find message level response makes the most sense to introduce in this PR, as operation level response is more of a nice-to-have feature. I see this because we need to be able to describe for a specific message what the desired response is, yes it is a bit cumbersome to define layer responses, but I think that feature belongs in its own issue, once the core issue has been resolved.
What do you think about this? 🙂
|
@fmvilas from my point of view this is |
|
Yup. Thanks for noticing, @smarek. |
|
Proposal shared with the wider community:
|
|
Thanks for @smarek for the interesting proposal. Reading through I had two thoughts:
In that way, limiting this functionality only to request reply amongst two consumers is perhaps not generic enough. It might be worth exploring how to handle choreography within the spec that handles "defining multi-directional conversations abstractly: Not just request/reply, but also scatter/gather patterns, saga patterns, etc.", as @clemensv puts it. |
|
@jmenning-solace thank you for discussion as well
|
|
Kudos, SonarCloud Quality Gate passed!
|
| @@ -1018,6 +1039,7 @@ Field Name | Type | Description | |||
| <a name="messageObjectBindings"></a>bindings | [Message Bindings Object](#messageBindingsObject) \| [Reference Object](#referenceObject) | A map where the keys describe the name of the protocol and the values describe protocol-specific definitions for the message. | |||
| <a name="messageObjectExamples"></a>examples | [Map[`string`, `any`]] | An array of key/value pairs where keys MUST be either **headers** and/or **payload**. Values MUST contain examples that validate against the [headers](#messageObjectHeaders) or [payload](#messageObjectPayload) fields, respectively. Example MAY also have the **name** and **summary** additional keys to provide respectively a machine-friendly name and a short summary of what the example is about. | |||
| <a name="messageObjectTraits"></a>traits | [[Message Trait Object](#messageTraitObject) | [Reference Object](#referenceObject)] | A list of traits to apply to the message object. Traits MUST be merged into the message object using the [JSON Merge Patch](https://tools.ietf.org/html/rfc7386) algorithm in the same order they are defined here. The resulting object MUST be a valid [Message Object](#messageObject). | |||
| <a name="messageObjectResponse"></a>response | [[Message Object](#messageObject) | [Reference Object](#referenceObject)] | A OPTIONAL definition of the message that the other party can expect to receive as a result to publish operation using this Message Object or as a result to receiving this Message Object in subscription. oneOf is allowed here to specify multiple messages, however, **a message MUST be valid only against one of the referenced message objects.** | |||
There was a problem hiding this comment.
Have you thought about whether you want to target this PR towards 2.x versions or 3.x? 🤔
Version 2.x
If you choose to target the 2.x version (I assume this is your intention), you will run into a few perspective issues that will need to be considered, let me try to explain it (this blog post explains the operation confusion). This perspective issue is also why at least I, have not tried to push the feature yet, as I could not figure out the meaning of response in this context 😅
Current perspective
Let's consider the two operation keywords (publish and subscribe) and what it means when reading the spec file:
channels:
basic:
publish:
message:
payload:
response:
payload:From the spec perspective, it means the following:
Others may
publishto the channelbasicwith message payloadx. Because I (the application) subscribe to the channelbasicexpecting a message payloadx.
channels:
basic:
subscribe:
message:
payload:
response:
payload:From the spec perspective, it means the following:
Others may
subscribeto the channelbasicexpecting a message payloadx. Because I (the application) publishes to the channelbasicwith the message payloadx.
Response keyword
Now, let's try to use the response keyword, what would it mean in conjunction with the publish operation keyword? The only way I see is that it means:
Others may
publishto the channelbasicwith message payloadxand expect aresponsewith message payloady. Because I (the application) subscribe to the channelbasicexpecting a message payloadxand I willresponsewith message payloady.
What would it mean in conjunction with the subscribe operation keyword? The only way I see is that it means:
Others may
subscribeto the channelbasicexpecting the message payloadxand I shouldresponsewith message payloady. Because I (the application) publish to the channelbasicwith the message payloadxand I will expect others toresponsewith message payloady.
As you can see operation keywords do not "match" with the response keyword. As in most cases respond makes more sense, but it switches based on what you try to define.
Maybe it is not a problem, however, I can't see how this confusion won't come up, especially considering how confusing the operation keywords already are 😅
What are your thoughts here?
Version 3.x
If you choose to target 3.x, we are relying on #618 to build upon any agreed change from there 🙂 @fmvilas already included an example of how it could be achieved, just using reply keyword instead of response.
There was a problem hiding this comment.
@jonaslagoni thank you for heads up, i intent to submit this against 2.x, because it's not major change, and can be ported forward to 3.x as well, but I fear that 3.x compliance among various tools and implementations is too far away for my goal, to get the RFC into production/real-live usage, as soon as possible.
Also thanks for detailed introspection in taxonomy of specification, the perspective is in my opinion:
basic client -> server scenario
I (the application) publish the message payload
xon channelbasicand I may expectresponse message payload ywhich can be correlated with message payloadxi've sent.
basic server -> client scenario
I (the application) may provide
response message payload yto every message payloadxserver publishes on channel i've previously subscribed to
basic multi-user scenario (eg. messaging server)
I (the application) subscribe to channel
basicand I may provideresponse message payload xto every received payloadxwhich might originate from server (service/channel) directly or from other clients on the same hub (not sure this term is understandable in context)
And for the linked #618, i actually prefer terms in a physics kind-of way, where action / reaction and request / response are more clear to me than message / reply or request / reply, but I'd definitely prefer consensus on the taxonomy before this gets into 2.x and/or 3.x, since it wouldn't make any sense to change the verb between major versions of specs
|
@smarek Could you please take a look at my comment? I'd like to hear from you and see how we can evolve the proposed model in order to support your requirements. As this proposal introduces breaking changes, they should be part of AsyncAPI v3.0.0. Thank you! |
|
To me, the channels:
basic:
exchanges:
fetchThing:
receive:
# ...
respond:
# ...
publish:
# ...That is to say:
This introduces a new, optional field of the Some semantic considerations:
|
|
I'd like to suggest another way of documenting request-reply messages: Instead of Documenting them on the request message (or seperately on the channel) moving the Documentation to the replies can have some benefits. channels:
requestData:
publish:
message:
$ref: '#/components/messages/requestSpecialData'
returnChannel:
subscribe:
message:
$ref: '#/components/messages/specialDataResponse'
trigger:
on: message
channel: requestData
message:
$ref: '#/components/messages/requestSpecialData'The samantic of this would roughly be: If a benefits:
Disadvantages:
I don't have a qood answer for where to put these triggers. In the current spec they should be as close as possible to the triggered messages, because we only have the two operations publish and subscribe per channel that need to describe all messages on that channel. If a channel can have any number of operations that can be specified as send or receive operations, then triggers should be specified on these operations (then we can also have triggers for when en operation fails with an error). If multiple operations per channel are allowed, then messages that have the same purpose can be grouped into a single operation that can be used to document these messages. Operations would then be logical operations that can be performed on a channel, rather than "physical" operations the channel itself allows (publish/subscribe or send/receive) like they are now. (The proposal in #618 already allows multiple operations per channel.) I believe that triggers are a better fit for async messaging based apis because they can capture the event based or reactive behaviour of such apis. They are also more flexible then a specific solution to document request-reply message pairs and can be extended in the future. |
|
Would this work well if we wanted to describe a JSON RPC 2.0 Api ? Here's an example request: And here's an error example: |
|
Another thing to consider is that we have some subscribers that ignore messages if there is not some kind of "reply to" address provided. e.g. if I have send a message to That being said, I'd like a way to specify that the subscriber will not perform any action if no reply address is given (e.g. |
Yes, however missing "id" param in error reply is problematic, since the reply cannot be easily linked with the request (which I'd expect in async reply, and which is understandably not required when response is provided synchronously, eg. the response is expected before another request occurs) |
I agree, but that should be in my opinion part of different RFC, since it describes topic/address related flow, not just "what kind of response can subscriber expect for given request/message" |
|
@buehlefs thank you, i don't think i will adopt your proposal in this RFC, since triggers are complex topic on its own. That being said how about channels:
basic:
message:
oneOf:
- $ref: '#/components/messages/ping'
responseBehavior: 'mandatory' # possible values (the list can be expanded) 'mandatory', 'optional', 'success-only', 'error-only', 'no-response'
response:
oneOf:
- $ref: '#/components/messages/unauthorized'
components:
messages:
ping:
payload:
$ref: '#/components/schemas/ping'
response:
$ref: '#/components/messages/pong'This way the reaction (response) of the other party could be expected without going much into detail of business-logic of the application |
|
And at last, as reaction to #94 (comment) by @derberg and other mentions of this topic, seems like the general community consensus is that this RFC contains breaking changes and should not be based against 2.x but 3.x version of AsyncAPI, where others and more complex topics are currently being addressed/discussed. I'm not able nor willing to join all the discussions. So if my understanding is correct, please close this PR and address the request/response needs in other way (more compliant with all the other changes designed in 3.x), i will not champion this topic in 3.x version. If it's not (meaning request/reply should be part of 2.x), I vote to not complicate the topic more here and provide basic support of request/reply as already presented in this PR, for something this trivial, the discussion is taking too long and i'm losing interest. References:
Please let me know, and if appropriate, you can directly close this PR |
|
First @smarek great job so for and i am totally with your. There is a need to describe the response well! Describe it unrelated would make the schema very confusing, as well if it is large (as i face it often). A little about my use caseI use messaging solution like JMS, MQTT, JCSMP, .... Where is see problemsWhere i see problems with your suggestion, there there is no possibility to define:
I dont think "responseBehavior" is an good idea:
Here my suggestion:An attribute: "channels[].subscribe.responseChannel" pointing to a message channel, can be the same but dont have to. Example for "Options A" channels:
tms/monitoring/monalesy/p/v1/serviceDesc/request:
subscribe:
traits:
- $ref: '#/components/operationTraits/solace'
message:
$ref: '#/components/messages/ping'
responseChannel: 'tms/monitoring/monalesy/p/v1/serviceDesc/inbox/{correlationId}'
tms/monitoring/monalesy/p/v1/serviceDesc/inbox/{correlationId}:
description:
The reply topic needs to start with api name "tms/monitoring/monalesy/p/v1/serviceDesc" to not colide with other application.
The reply topic needs to match a schema will not be blocked by ACL "tms/monitoring/monalesy/p/v1/serviceDesc/inbox/*".
parameters:
correlationId:
$ref: '#/components/schemas/correlationId'
response:
traits:
- $ref: '#/components/operationTraits/solace'
message:
$ref: '#/components/messages/pong'
components:
messages:
ping:
traits:
- $ref: '#/components/messageTraits/requestHeader'
payload:
$ref: '#/components/schemas/ping'
pong:
traits:
- $ref: '#/components/messageTraits/responseHeader'
payload:
$ref: '#/components/schemas/pong'
messageTraits:
requestHeader:
headers:
type: object
required:
- correlationId
properties:
correlationId:
$ref: '#/components/schemas/correlationId'
responseHeader:
headers:
type: object
properties:
messageIndex:
type: int
minimum: 0
messageCount:
type: int
minimum: 1Example for "Options B" channels:
tms/monitoring/monalesy/p/v1/serviceDesc/request:
subscribe:
traits:
- $ref: '#/components/operationTraits/solace'
message:
$ref: '#/components/messages/ping'
responseChannel: 'tms/monitoring/monalesy/p/v1/serviceDesc/request'
response:
traits:
- $ref: '#/components/operationTraits/solace'
message:
$ref: '#/components/messages/pong'
components:
messages:
ping:
traits:
- $ref: '#/components/messageTraits/requestHeader'
payload:
$ref: '#/components/schemas/ping'
pong:
traits:
- $ref: '#/components/messageTraits/responseHeader'
payload:
$ref: '#/components/schemas/pong'
messageTraits:
requestHeader:
headers:
type: object
required:
- correlationId
- replyTo
properties:
correlationId:
type: string
format: uuid
replyTo:
type: string
examples:
- tms/monitoring/monalesy/p/v1/serviceDesc/inbox/your-host-name/your-process-uuid
responseHeader:
headers:
type: object
required:
- correlationId
properties:
correlationId:
type: string
format: uuid
messageIndex:
type: int
minimum: 0
messageCount:
type: int
minimum: 1 |
|
super important update to folks interested in this topic. we will host a dedicated discussion next Monday -> asyncapi/community#352 |
|
This pull request has been automatically marked as stale because it has not had recent activity 😴 It will be closed in 120 days if no further activity occurs. To unstale this pull request, add a comment with detailed explanation. There can be many reasons why some specific pull request has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model. Let us figure out together how to push this pull request forward. Connect with us through one of many communication channels we established here. Thank you for your patience ❤️ |
|
Closing as #847 has been merged 🎉 |
Title: "Request/Response paradigm support in both operation and message objects"
Related issue(s): #55 #94 #558
Champion: @smarek
Since nobody yet took the opportunity, i'm gonna try to champion this Feature RFC.
Explanation of Problem & Solution:
Very common type of operation is currently missing in specification per discussion in referenced (related) issues. Operation that results in direct or indirect response to the caller.
The most basic example is publishing a message, where the result of publish operation should be known as soon as possible to the caller. Eg. invoking operation on given channel will return the specific message (or oneOf messages) right away to the caller.
The complicated example is correlation of the request and response, where publish/subscribe operation with given message will result in the specific message to be delivered back to the caller via different channel.
Solution has two parts:
Which allows for "Acknowledge" messages, "HTTP Request/Response" correlation and foremost to better describe (and for api document reader understand) the situations where one might receive specific message
Illustrative examples:
I'll limit myself to two examples, as various are already discussed in related (referenced) issues.
- HTTP standard expects the caller to consume the response headers and possibly a response payload, if given HTTP verb allows it.
- This feature would allow to specify response headers and json body that would represent the result of HTTP GET, POST, PUT or other http operation
- SignalR interface specifies two-way methods in rpc-like manner, where client or server can invoke operations (interface methods/functions) on each other, where each operation can take arguments and can produce any kind of response (be it scalar value or complex objects), the transport is JSON over WebSocket
- This feature allows the SignalR protocol to correctly define the messages that client/server can receive as a result of invoking operation on the other party OR invoking operation with specific message type
Also mentioned should be usage of current
x-responseextension https://www.asyncapi.com/blog/websocket-part2#describe-responses---specification-extensionsIdentification of potential concerns, challenges, and drawbacks:
- I actually think it is desired and it should be concern of user/developer whether the response message chaining is used correctly or not
- This is in my opinion necessary to support as wide spectrum of bindings/protocols as possible, some protocols produce different kind of messages on single channel item operation (eg.
getNextOfAnyTypekind of operations) and some protocols use the invoke/getResult paradigm where the response message is always directly linked to the invoked operation and "request message" used in suchresultfields OPTIONAL is not long-lasting solution and will not drive the community to use the option to make their API documentations more strong-typed, and I'm honestly not sure whether there is anything we can do about it