Skip to content

2. Handlers, Preprocessors and Services

Jump to bottom
Juliette Regimbal edited this page Oct 2, 2024 · 1 revision

Handlers, Preprocessors and Services: What are they?

The server architecture is not monolithic. Instead, it is built up of various components running as Docker services (in swarm mode) or Docker containers (regularly). These components fill various roles in the process of creating renderings to be sent to the user, but they all communicate with each other using HTTP* and are stateless. They should also all follow typical coding guidelines for the language they are written in, for example PEP 8 for Python. More information about the architecture can be found in our W4A'22 communication paper.

*The SuperCollider service uses OSC instead, but this is an exception as SuperCollider does not have HTTP support.

Preprocessors

Preprocessors are components that add additional data to a request sent by the client. This data can be as simple as checking if the URL of the page matches some pattern (e.g., is it from CNN?) or as complicated as performing some ML process on the image. Whatever a preprocessor does it MUST:

  1. Listen for POST requests at the /preprocessor route where the body is the previously mentioned request JSON.
  2. Reply to these POST requests with a properly-formatted response in the event the preprocessor action succeeds.
    • If the request is improperly formatted, the preprocessor must respond with a 400 status and SHOULD report the error in the body of the response.
    • The preprocessor should also check if the response generated passes validation. If it performs this check and the response fails, it must respond with a 500 status indicating the issue occurred in the preprocessor.
    • Finally, there are cases where everything is correct, but the preprocessor shouldn't run. In this case, a 204 status should be returned with no body. Cases like this are elaborated on below.
  3. The response must be generated and sent within 15 seconds as of 2021-05-25. If a response is not received in this time, the connection is broken and the preprocessor data will not be available later on.

Handlers

Handlers are component that, using the preprocessor data and the request itself, generate renderings that can be selected by the user. These renderings are accompanied by a confidence score (0–100%) indicating how likely it is that the rendering correctly conveys information on the source image. 0% indicates that the handler is absolutely certain any rendering is incorrect, while 100% should be reserved for manually created renderings for certain media (e.g., a handler that displays a custom map of the metro network and nothing else). Handlers have similar requirements to preprocessors in that they must:

  1. Respond to POST requests at /handler containing the request JSON with a well-formatted response.
  2. If the request JSON is not properly formatted, it must respond with a 400 error.
  3. If for any other reason the handler cannot generate a proper response (i.e., a failure occurs), it must respond with a 500 error.
  4. If the handler can't actually generate anything, it should respond with an empty list of renderings.
  5. The handler should respond as quickly as possible despite not having a fixed timeout period. It is likely that at some point handlers that do not respond quickly will be cancelled. Since many handlers will run in parallel, resource contention will be an issue. So, it may be most efficient to emphasize quickly deciding that the handler cannot produce a good rendering, and terminate as early as possible, rather than generating an entire rendering that will have a low confidence score.

Services

Services perform fixed convenience jobs to multiple preprocessors and handlers. A service should essentially do one kind of thing well, like TTS, that is likely to be reused. Services are less strictly defined than preprocessors and handlers, but still should follow some guidelines:

  1. For ease of use, they should respond to requests on port 80.
  2. Each service should define the requests and responses it deals with.
    • If possible, these should be shared across a type of service to permit easily switching (e.g., all TTS services use a shared interface).
  3. The route for requests should be /service/[generic type]/[specific command]. E.g., creating a set of TTS audio uses /service/tts/segment-tts.

Docker Compose Configuration

This section applies to preprocessors and handlers. Services also run in Docker, but do not need this configuration as they don't need to be advertised to a central service. The preprocessors and handlers need to be visible to a receiver component that actually receives the request from the client, passes this to other components in order, and enforces schema compliance.

To find these other components, the receiver connects to the Docker daemon and looks for services/containers with certain labels. For preprocessors, the preprocessor must contain the label ca.mcgill.a11y.image.preprocessor where its value is a number. This number is used to sort the preprocessors. So all preprocessors with this label set to 1 will run before those set to 2. This is used to allow preprocessors to ensure access to information from other preprocessor processes. For handlers, this label is ca.mcgill.a11y.image.handler which MUST be set to the string "enable". Handlers run in parallel, and so no ordering is required.

Both handlers and preprocessors are assumed to be running on port 80 by default. If they do not use this port, then they must specify the port using the ca.mcgill.a11y.image.port label. So, a preprocessor that relies on data from another preprocessor and listens to port 8080 would have the following labels fragment in a Docker compose file:

labels:
    ca.mcgill.a11y.image.preprocessor: 2
    ca.mcgill.a11y.image.port: 8080

Preprocessor Chaining Best Practices

As explained on above, preprocessors run in an order and must return 204 quickly if they cannot do anything with a request. While this conclusion can sometimes be made from the original request (e.g., this preprocessor only works with images from a certain site), often this requires looking at the output of a previously run preprocessor. It is a goal of IMAGE to keep preprocessors (and other components) modular so they can be reused and repurposed on another server instance. This other instance may not have the same goals or requirements as our instance. To avoid someone needing to edit our code to adapt to a particular use case, we require the following to be done for preprocessors that use the output of another preprocessor:

  • If the previous preprocessor's output is available and indicates conditions where the current preprocessor should run, it should run normally.
  • If the previous preprocessor's output is available and indicates conditions where the current preprocessor should not run, it should return 204.
  • If the previous preprocessor's output is not available and its data is not strictly necessary for the current preprocessor to run, it should run normally.
  • If the previous preprocessor's output is not available and its data is strictly necessary for the current preprocessor to run, it should return 204.

In no case should a preprocessor return a 400 or 500 error when a request was properly formatted and no operational errors had occurred.

Clone this wiki locally