Bad operationId to error status code #278

jfinkhaeuser · 2016-09-07T12:44:11Z

Adds the ability to turn bad or missing operationIds into run time errors returning a defined status code, e.g. 501 ("Not Implemented").

This change is largely useful for testing purposes.

When (black-box) testing implementations, it is not yet know whether a given implementation matches the given Swagger/OpenAPI specs. Especially during development, partial implementations might exist and test well, but some endpoints might still be missing from the implementation. This change lets connexion serve such partial implementations, and lets testers write expectations about missing functionality (i.e. results in a 501 error).

Note that this PR includes #274, #275, #276 and #277, and in fact depends on those changes. I would recommend merging the others first (or not, as it were) before merging this one.

coveralls · 2016-09-07T12:49:03Z

Coverage remained the same at 100.0% when pulling d755859 on jfinkhaeuser:bad-operation-id-to-5xx into 52ec049 on zalando:master.

coveralls · 2016-09-07T13:23:43Z

Coverage remained the same at 100.0% when pulling 70d9d9d on jfinkhaeuser:bad-operation-id-to-5xx into 52ec049 on zalando:master.

rafaelcaricio · 2016-09-08T15:04:07Z

tox.ini

@@ -3,7 +3,7 @@ max-line-length=120
 exclude=connexion/__init__.py

 [tox]
-envlist=pypy,py27,py34,py35,py35-flask0.10.1,isort-check,flake8
+envlist=py27,py34


Please change this back. Not sure if you committed that change by mistake.

Oh yes, very much so! Sorry about that!

rafaelcaricio · 2016-09-08T15:06:07Z

Rebase this PR please.

jfinkhaeuser · 2016-09-08T15:35:28Z

Yep. Probably tomorrow.

- Handle bad and missing operationIds with a well-defined error

…from the module with error case, but the result should be the same.

part of the PR earlier.

next up: the implementation.

complain about duplicate names.

resolver_error_handler is provided, try to add that as an operation with a randomized endpoint name. Otherwise, behave as before.

callback that's creating actual handler operations. That's more in line with the overall code structure.

ResolverError occurrences into a ResolverErrorHandler that returns the error code given.

and re-raises its exception information if exceptions relating to name resolution are raised at all. It still converts the meaningless ValueError and AttributeError into a ResolverError.

coveralls · 2016-09-09T09:59:53Z

Coverage remained the same at 100.0% when pulling 37154d3 on jfinkhaeuser:bad-operation-id-to-5xx into ec1ad38 on zalando:master.

coveralls · 2016-09-09T10:06:24Z

Coverage remained the same at 100.0% when pulling 70c5bb9 on jfinkhaeuser:bad-operation-id-to-5xx into ec1ad38 on zalando:master.

begin with.

coveralls · 2016-09-12T12:00:24Z

Coverage remained the same at 100.0% when pulling 328aa51 on jfinkhaeuser:bad-operation-id-to-5xx into ec1ad38 on zalando:master.

See #274 for discussion.

coveralls · 2016-09-13T13:43:22Z

Coverage remained the same at 100.0% when pulling 0bc04c5 on jfinkhaeuser:bad-operation-id-to-5xx into f8c7a68 on zalando:master.

rafaelcaricio · 2016-09-13T13:46:06Z

tests/test_api.py

    api = Api(TEST_FOLDER / "fixtures/user_module_loading_error/swagger.yaml", "/api/v1.0",
              {'title': 'OK'}, debug=True)
    assert api.specification['info']['title'] == 'OK'

-    api = Api(TEST_FOLDER / "fixtures/module_not_implemented/swagger.yaml", "/api/v1.0",
+    api = Api(TEST_FOLDER / "fakeapi/missing_op_id.yaml", "/api/v1.0",


Please use this swagger file from the /fixtures dir.

rafaelcaricio · 2016-09-13T13:50:38Z

@jfinkhaeuser Would be nicer to have this as an option, it could be called stub_missing_operations (or something similar) so it would be aligned with what I tried to do in #284. In my PR I used 400 status code which now I see would be better to be 501. I am willing to change the #284 to use this PR.

jfinkhaeuser · 2016-09-13T13:57:32Z

Makes sense as an option to me, yes.

Whether you change #284 or I change this one, I don't mind particularly :)

coveralls · 2016-09-13T13:58:27Z

Coverage remained the same at 100.0% when pulling ecd7d3a on jfinkhaeuser:bad-operation-id-to-5xx into f8c7a68 on zalando:master.

rafaelcaricio · 2016-09-13T14:02:44Z

It is preferably to this change to be in this PR since it is where this behavior is going to be merged.

jfinkhaeuser · 2016-09-13T14:04:58Z

Alright, I'll look into it.

coveralls · 2016-09-13T14:09:31Z

Coverage remained the same at 100.0% when pulling 6476c03 on jfinkhaeuser:bad-operation-id-to-5xx into f8c7a68 on zalando:master.

coveralls · 2016-09-13T14:30:18Z

Coverage remained the same at 100.0% when pulling 74cec24 on jfinkhaeuser:bad-operation-id-to-5xx into f8c7a68 on zalando:master.

rafaelcaricio · 2016-09-13T14:40:53Z

tests/test_api.py

+            {'title': 'OK'})
+
+    # Debug mode should ignore the error
+    api = Api(TEST_FOLDER / "fakeapi/bad_specs.yaml", "/api/v1.0",


Could you please move those specs to a proper directory within /fixtures?

Ah, yeah, the bad specs I missed. One moment.

jfinkhaeuser · 2016-09-13T14:50:25Z

So now I've re-familiarized myself with everything, I've got to ask: nice to have as an option to what?

Right now, it's an option to add_api, i.e. if resolver_error is an int, that's the status code that'll be used. If none, the default behaviour is used.

It's relatively hard adding this functionality to a StubResolver as in #284. There's a bunch of information missing in Resolver to make that easy, and IMHO it's not a resolver's purpose to make up things that didn't resolve.

So I've put it into Api instead, which makes a lot more sense to me. add_api gets the option for convenience.

coveralls · 2016-09-13T14:59:18Z

Coverage remained the same at 100.0% when pulling fa328f7 on jfinkhaeuser:bad-operation-id-to-5xx into f8c7a68 on zalando:master.

rafaelcaricio · 2016-09-13T15:52:22Z

Yes, you are right! I looked through your code here after posting my comment and I see that it is already an option. I will do a final review before merging it.

rafaelcaricio · 2016-09-13T20:02:43Z

connexion/app.py

@@ -96,7 +96,7 @@ def common_error_handler(exception):

    def add_api(self, swagger_file, base_path=None, arguments=None, auth_all_paths=None, swagger_json=None,
                swagger_ui=None, swagger_path=None, swagger_url=None, validate_responses=False,
-                strict_validation=False, resolver=Resolver()):
+                strict_validation=False, resolver=Resolver(), resolver_error=None):


It is not very clear that resolver_error is expected to be a HTTP status code. Would be nice to have a more friendly name here.

rafaelcaricio · 2016-09-13T20:04:32Z

👍

This was referenced Sep 7, 2016

Use generated operations Trax-air/swagger-tester#27

Merged

Add ability to add random string to endpoint names #275

Merged

rafaelcaricio reviewed Sep 8, 2016
View reviewed changes

jfinkhaeuser added 20 commits September 9, 2016 11:52

- Add test cases for bad and missing operationIds

a0071d6

- Handle bad and missing operationIds with a well-defined error

Add test cases for missing and bad operationIds - slightly different …

ad0a908

…from the module with error case, but the result should be the same.

YAML file for the previous test case

1648dd5

ImportError should also be translated into ResolverError

1914e56

Obviously, ImportError no longer falls through. That should have been

875efbf

part of the PR earlier.

Add test cases for the resolver_error flag. That's what it should do,

3cd04f8

next up: the implementation.

Add ability to add random string to endpoint names so that flask doesn't

d27264d

complain about duplicate names.

Mhh, that change should not have been there.

7864a9d

Fix flake8 complaints

2cae3b1

Add ability for Operation to randomize endpoint name (off by default)

44d4140

Improve Api test cases

92f51ea

Move ResolverError over to exceptions where it seems to be long.

8c55654

Give the same string representation as other exception classes

4d706d5

If a ResolverError is raised during the addition of operations, and a

1e786d9

resolver_error_handler is provided, try to add that as an operation with a randomized endpoint name. Otherwise, behave as before.

Instead of making resolver_error_handler a magic dict, make it a

eacfe47

callback that's creating actual handler operations. That's more in line with the overall code structure.

Add ResolverErrorHandler, analogous to AuthErrorHandler

34ee0d1

When adding an API, allow specifying an error code for turning

4203f4c

ResolverError occurrences into a ResolverErrorHandler that returns the error code given.

Add YAML for bad specs test

f50da76

Report more about the ResolverError details.

9196b38

Makes sure #250 is still working. This explicitly handles ImportError

37154d3

and re-raises its exception information if exceptions relating to name resolution are raised at all. It still converts the meaningless ValueError and AttributeError into a ResolverError.

Merge branch 'reuse-endpoint-names' into bad-operation-id-to-5xx

70c5bb9

Fix isort errors. Some of these look like they're not from my branch to

328aa51

begin with.

jfinkhaeuser added 2 commits September 13, 2016 15:37

Merge branch 'fix-missing-operation-id' into bad-operation-id-to-5xx

9d4a40e

See #274 for discussion.

Merge branch 'master' into bad-operation-id-to-5xx

0bc04c5

rafaelcaricio reviewed Sep 13, 2016
View reviewed changes

Fix tests after merge

ecd7d3a

Fix isort errors

6476c03

jfinkhaeuser added 2 commits September 13, 2016 16:12

This file shouldn't have been added.

7b7b11d

Fixes after running & fixing TOXENV=isort-check tox

74cec24

rafaelcaricio reviewed Sep 13, 2016
View reviewed changes

Move bad_specs fixtures to its own fixtures folder

fa328f7

rafaelcaricio reviewed Sep 13, 2016
View reviewed changes

rafaelcaricio merged commit 58c99a0 into spec-first:master Sep 13, 2016

rafaelcaricio mentioned this pull request Sep 16, 2016

Support for mocking API based on spec #292

Closed

allan-silva mentioned this pull request Mar 11, 2017

B1325377 - Public API to Balrog's database mozilla-releng/balrog#276

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bad operationId to error status code #278

Bad operationId to error status code #278

jfinkhaeuser commented Sep 7, 2016

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 7, 2016 •

edited

Loading

rafaelcaricio Sep 8, 2016

jfinkhaeuser Sep 8, 2016

rafaelcaricio commented Sep 8, 2016

jfinkhaeuser commented Sep 8, 2016

coveralls commented Sep 9, 2016 •

edited

Loading

coveralls commented Sep 9, 2016 •

edited

Loading

coveralls commented Sep 12, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio Sep 13, 2016

rafaelcaricio commented Sep 13, 2016 •

edited

Loading

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio commented Sep 13, 2016

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio Sep 13, 2016

jfinkhaeuser Sep 13, 2016

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio commented Sep 13, 2016 •

edited

Loading

rafaelcaricio Sep 13, 2016

rafaelcaricio commented Sep 13, 2016

Bad operationId to error status code #278

Bad operationId to error status code #278

Conversation

jfinkhaeuser commented Sep 7, 2016

coveralls commented Sep 7, 2016 • edited Loading

coveralls commented Sep 7, 2016 • edited Loading

rafaelcaricio Sep 8, 2016

Choose a reason for hiding this comment

jfinkhaeuser Sep 8, 2016

Choose a reason for hiding this comment

rafaelcaricio commented Sep 8, 2016

jfinkhaeuser commented Sep 8, 2016

coveralls commented Sep 9, 2016 • edited Loading

coveralls commented Sep 9, 2016 • edited Loading

coveralls commented Sep 12, 2016 • edited Loading

coveralls commented Sep 13, 2016 • edited Loading

rafaelcaricio Sep 13, 2016

Choose a reason for hiding this comment

rafaelcaricio commented Sep 13, 2016 • edited Loading

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 • edited Loading

rafaelcaricio commented Sep 13, 2016

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 • edited Loading

coveralls commented Sep 13, 2016 • edited Loading

rafaelcaricio Sep 13, 2016

Choose a reason for hiding this comment

jfinkhaeuser Sep 13, 2016

Choose a reason for hiding this comment

jfinkhaeuser commented Sep 13, 2016

coveralls commented Sep 13, 2016 • edited Loading

rafaelcaricio commented Sep 13, 2016 • edited Loading

rafaelcaricio Sep 13, 2016

Choose a reason for hiding this comment

rafaelcaricio commented Sep 13, 2016

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 7, 2016 •

edited

Loading

coveralls commented Sep 9, 2016 •

edited

Loading

coveralls commented Sep 9, 2016 •

edited

Loading

coveralls commented Sep 12, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio commented Sep 13, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

coveralls commented Sep 13, 2016 •

edited

Loading

rafaelcaricio commented Sep 13, 2016 •

edited

Loading