Response validation kicks in before Flask's after_request hook or WSGI middlewares that modify the response

[dirkschneemann]

Description

Adding validate_responses=True to the app.add_api() method when initializing my application will validate all returned responses against the response models from my OpenAPI specification.

However, if I use either a custom WSGI middleware or Flask's after_request hook to modify the response object before it gets returned, this is not reflected during the validation. Consequently, validation might fail even though my response is actually matching the specification.

Expected behaviour

Response validation kicks in after all handlers that could possibly modify the response have been run.

Actual behaviour

Response validation kicks in after all my application logic has been executed but before custom WSGI middlewares or Flask's after_request hook had a chance to modify the response.

Steps to reproduce

specification.yml

openapi: 3.0.2
info:
  title: Some API
  version: Some version
paths:
  /:
    get:
      operationId: main.magic
      responses:
        200:
          description: OK
          content:
            text/plain:
              schema:
                type: string
          headers:
            Custom-Header:
              schema:
                type: string

Using a WSGI middleware:
main.py

import connexion


class CorsHeaderMiddleware(object):
    def __init__(self, app):
        self.app = app

    def __call__(self, environ, start_response):
        def custom_start_response(status, headers, exc_info=None):
            headers.append(('Custom-Header', 'Some value'))
            return start_response(status, headers, exc_info)

        return self.app(environ, custom_start_response)


def magic():
    return 'Hooray'


connexion_app = connexion.FlaskApp(__name__, specification_dir='.')
connexion_app.add_api('specification.yml', validate_responses=True)
connexion_app.app.wsgi_app = CorsHeaderMiddleware(connexion_app.app.wsgi_app)
connexion_app.run(port=8000)

Using Flask's after_request hook:
main.py

import connexion

def set_custom_header_on_response(response):
    response.headers['Custom-Header'] = 'Some value'
    return response

connexion_app = connexion.FlaskApp(__name__, specification_dir='.')
connexion_app.add_api('specification.yml', validate_responses=True)
connexion_app.app.after_request(set_custom_header_on_response)
connexion_app.run(port=8000)

Testing (same result in both examples):

curl -i -XGET localhost:8000/
HTTP/1.0 500 INTERNAL SERVER ERROR
Content-Type: application/problem+json
Content-Length: 200
Custom-Header: Some value
Server: Werkzeug/0.14.1 Python/3.5.2
Date: Fri, 23 Nov 2018 17:53:29 GMT

{
  "detail": "Keys in header don't match response specification. Difference: Custom-Header",
  "status": 500,
  "title": "Response headers do not conform to specification",
  "type": "about:blank"
}

As one can see, the custom header has been set despite the validation error.

Additional info:

Output of the commands:

• python --version: Python 3.5.2
• pip show connexion | grep "^Version\:": Version: 2.0.0

[halfdan]

Experiencing the same issue. We have an etag middleware in place which adds etag headers using after_request. Our OpenAPI spec lists the etag cache-control headers as required. validate_responses is failing because it gets evaluated before the after_request handlers which makes it effectively unusable for us.

[RobbeSneyders]

This is due to how Connexion works at its core. Connexion adds its magic by wrapping the view functions with decorators, so the request flow is ->app->connexion->view_function->connexion->app (see the Connexion architecture). To be able to validate Flask's after_request or middleware, Connexion would have to act like a middleware itself.

This is something we're looking into for Connexion 3.0, however it could come with the trade off of losing some other features when running in middleware mode.

[RobbeSneyders]

No longer the case since #1591