Document HTTP gateway (#131)

Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
LangStream · Nov 2, 2023 · 3a5cc90 · 3a5cc90
1 parent f0dffb9
commit 3a5cc90
Show file tree

Hide file tree

Showing 5 changed files with 143 additions and 15 deletions.
diff --git a/SUMMARY.md b/SUMMARY.md
@@ -42,6 +42,7 @@
 * [Expression Language](building-applications/expression-language.md)
 * [API Gateways](building-applications/api-gateways/README.md)
   * [Websocket](building-applications/api-gateways/websocket.md)
+  * [HTTP](building-applications/api-gateways/http.md)
   * [Message filtering](building-applications/api-gateways/message-filtering.md)
   * [Gateway authentication](building-applications/api-gateways/gateway-authentication.md)
 * [API Reference](building-applications/api-reference/README.md)

diff --git a/building-applications/api-gateways/README.md b/building-applications/api-gateways/README.md
@@ -1,16 +1,20 @@
 # API Gateways
 
-API gateways in LangStream are WebSocket endpoints that allow applications to interact with agents via message topics.
+API gateways in LangStream are HTTP/WebSocket endpoints that allow applications to interact with agents via message topics.
 
 Messages contain a key, a value, and headers for metadata. This metadata is used to connect clients to topics and can be used to [filter messages.](message-filtering.md)
 
 LangStream applications can have three different gateway types:
 
 **Producer** gateways are clients that create a message and write it to a topic.&#x20;
 
-**Consumer** gateways are clients that watch a topic for new messages.&#x20;
+**Consumer** gateways are clients that watch a topic for messages from a given position. This gateway can only be accessed via WebSocket. &#x20;
 
-**Chat** gateways create a producer client ("questions-topic") and a consumer client ("answers-topic") across one connection, optimized for "chat" style interactions between client and server.
+**Chat** gateways create a producer client (`questions-topic`) and a consumer client (`answers-topic`) across one connection, optimized for "chat" style interactions between client and server. This gateway can only be accessed via WebSocket.&#x20;
+
+**Service (topics)** gateways create a producer client (`input-topic`) and a consumer client (`output-topic`), writing a single message and awaiting for the receivement of another message. This gateway can only be accessed via HTTP.&#x20;
+
+**Service (agents)** gateways send requests directly to a `service` agent and return the response to the client. This gateway can only be accessed via HTTP.
 
 ### Example gateways
 
@@ -29,8 +33,14 @@ gateways:
   - id: "chat"
     type: chat
     chat-options:
-      answers-topic: output-topic
-      questions-topic: input-topic
+      questions-topic: questions-topic
+      answers-topic: answers-topic
+
+  - id: "service"
+    type: service
+    service-options:
+      input-topic: questions-topic
+      output-topic: answers-topic
 ```
 
 The corresponding URLs to the gateways would be:
@@ -41,7 +51,12 @@ The corresponding URLs to the gateways would be:
 
 **Chat gateway:** ws://localhost:8091/consume/my-super-cool-tenant/some-application/chat
 
-The URL structure is ws://\<control-plane-domain>:\<api-gateway-port>/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id>
+**Service gateway:** http://localhost:8091/api/gateways/service/my-super-cool-tenant/some-application/service
+
+The URL structure is ws://\<control-plane-domain>:\<api-gateway-port>/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id> for WebSocket.
+
+The URL structure is http://\<control-plane-domain>:\<api-gateway-port>/api/gateways/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id> for HTTP.
+
 
 ### Gateways configuration
 

diff --git a/building-applications/api-gateways/gateway-authentication.md b/building-applications/api-gateways/gateway-authentication.md
@@ -36,12 +36,12 @@ gateways:
       provider: google
       configuration:
         clientId: "{{ secrets.google.client-id }}"
-    produceOptions:
+    produce-options:
       headers:
         - key: langstream-client-user-id
-        valueFromAuthentication: subject
+          value-from-authentication: subject
         - key: langstream-client-session-id
-          valueFromParameters: sessionId
+          value-from-parameters: sessionId
 
   - id: "bot-output"
     type: consume
@@ -52,17 +52,17 @@ gateways:
       provider: google
       configuration:
         clientId: "{{ secrets.google.client-id }}"
-    produceOptions:
+    produce-options:
       headers:
         - key: langstream-client-user-id
-        valueFromAuthentication: subject
+          value-from-authentication: subject
         - key: langstream-client-session-id
-          valueFromParameters: sessionId
+          value-from-parameters: sessionId
 ```
 
 #### Google authentication
 
-Set `provider:google` to use a Google client ID to authenticate LangStream gateway connections.
+Set `provider: google` to use a Google client ID to authenticate LangStream gateway connections.
 
 The Google field that is exposed for authentication is "subject".
 
@@ -82,7 +82,7 @@ export GOOGLE_CLIENT_ID=99840107278-4363876v0hker43roujaubqom5g07or8.apps.google
 
 #### Github authentication
 
-Set `provider:github` to use a GitHub login to authenticate LangStream gateway connections.
+Set `provider: github` to use a GitHub login to authenticate LangStream gateway connections.
 
 The Github field that is exposed for authentication is "login".
 

diff --git a/building-applications/api-gateways/http.md b/building-applications/api-gateways/http.md
@@ -0,0 +1,112 @@
+# HTTP
+
+You connect to a gateway via standard HTTP 1.1. While connecting to a gateway, the path holds pointers to the type of gateway, the tenant, the application id, and the gateway name.
+
+The URL structure is http://\<control-plane-domain>:\<api-gateway-port>/api/gateways/\<gateway-type>/\<tenant-name>/\<application-id>/\<gateway-id>.
+
+Authentication (`credentials,test-credentials`), options (`option:`) and parameters (`param:`) are expected to be in the query string. HTTP headers are ignored by the API Gateway service.
+
+
+## Produce messages
+
+To produce messages via HTTP, configure a `produce` gateway:
+```yaml
+
+gateways:
+  - id: "user-input"
+    type: produce
+    topic: "questions-topic"
+    parameters:
+    - sessionId
+```
+
+Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
+
+```bash
+curl -X POST --body '{"value": "hello"}' "http://localhost:8091/api/gateways/produce/my-tenant/my-app/user-input?param:sessionId=12543yusi1"
+```
+
+You can also use the LangStream CLI:
+
+```bash
+langstream gateway produce my-app user-input --protocol http -v '{"value": "hello"}' -p sessionId=12543yusi1
+```
+
+
+## Produce messages and wait for a message
+
+To produce messages via HTTP and wait for a message, configure a `service` gateway:
+```yaml
+
+gateways:
+  - id: "user-input-await"
+    type: service
+    parameters:
+    - sessionId
+    service-options:
+        input-topic: inputs
+        output-topic: results
+```
+
+Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
+
+```bash
+curl -X POST --body '{"value": "hello"}' "http://localhost:8091/api/gateways/service/my-tenant/my-app/user-input-await"
+```
+
+The timeout of the wait is the TCP timeout of the connection, which is usually 30 seconds (may vary depending on the http client).
+
+## Proxy service agent requests
+
+To proxy requests to a specific `service` agent via HTTP, configure a `service` gateway:
+
+```yaml
+gateways:
+  - id: "my-service"
+    type: service
+    parameters:
+    - sessionId
+    service-options:
+        agent-id: my-service-agent
+     
+```
+
+A service agent might look like this in the pipeline configuration: 
+```yaml
+pipeline:
+  - name: "Start my service"
+    id: my-service-agent
+    type: "python-service"
+    configuration:
+      className: example.ChatBotService
+```
+
+
+Once a gateway is configured, you can use whatever HTTP client you prefer to connect. This is an example with Curl:
+
+```bash
+curl -X POST \
+    --body '{"value": "hello"}' \
+    --H 'Authorization: Bearer XXX' \
+    "http://localhost:8091/api/gateways/service/my-tenant/my-app/my-service/the/custom/path?service-param-1=yes"
+```
+
+The final part of the URL, query string, HTTP method, and headers will be sent to the service agent.
+
+
+In the above case, the agent service will receive an equivalent request of:
+
+```
+curl -X POST \
+    --body '{"value": "hello"}' \
+    --H 'Authorization: Bearer XXX' \
+    "http://localhost:8000/the/custom/path?service-param-1=yes"
+```
+
+
+The `credentials`, `test-credentials`, `option:xx`, `param:xx` are stripped out from the routed requests.
+If the gateway has authentication enabled, it will be performed as for other gateways.
+
+POST, GET, DELETE and PUT are all supported.
+
+Leveraging the API gateway to expose your service solves authentication, HTTPS, high-availability, and scalability out of the box. 
diff --git a/building-applications/api-gateways/message-filtering.md b/building-applications/api-gateways/message-filtering.md
@@ -55,7 +55,7 @@ Chat gateways behave a little differently, because they open producers and consu
 
 ### headers
 
-<table><thead><tr><th width="244">Label</th><th width="113.33333333333331">Type</th><th>Description</th></tr></thead><tbody><tr><td>key</td><td>string</td><td>An identifier of the value. This can be thought of as the name. Allowed characters are a-zA-Z0-9._-</td></tr><tr><td>valueFromParameters</td><td>string</td><td>The mapped name to connect a querystring value in the Gateway URL to message headers. This must match a value set in the parameters collection.</td></tr><tr><td>valueFromAuthentication</td><td>string</td><td>The mapped name to connect a provided querysting value with the identity provider’s handshake value. See <a href="gateway-authentication.md">Gateway authentication.</a></td></tr></tbody></table>
+<table><thead><tr><th width="244">Label</th><th width="113.33333333333331">Type</th><th>Description</th></tr></thead><tbody><tr><td>key</td><td>string</td><td>An identifier of the value. This can be thought of as the name. Allowed characters are a-zA-Z0-9._-</td></tr><tr><td>value-from-parameters</td><td>string</td><td>The mapped name to connect a querystring value in the Gateway URL to message headers. This must match a value set in the parameters collection.</td></tr><tr><td>value-from-authentication</td><td>string</td><td>The mapped name to connect a provided querystring value with the identity provider’s handshake value. See <a href="gateway-authentication.md">Gateway authentication.</a></td></tr></tbody></table>
 
 ### Message offset