Support for Cerberus 1.1 #1001

bcrochet · 2017-03-21T00:14:52Z

(Based on original patch from dkellner)

This is a rather big change. I still decided to do a single commit, as
intermediate commits would be in a non-working state anyway.

Breaking changes:

keyschema was renamed to valueschema and propertyschema to
keyschema (following changes in Cerberus).
A PATCH on a document which misses a field having a default value will
now result in setting this value, even if the field was not provided
in the PATCH's payload.
Error messages for keyschema are now returned as dictionary.
Before: {'propertyschema_dict': 'propertyschema_dict'}
Now: {'keyschema_dict': {'AAA': "value does not match regex '[a-z]+'"}}
Error messages for type validations are different now (following
changes in Cerberus).
It is no longer valid to have a field with default = None and
nullable = False.
(see patch.py:test_patch_nested_document_nullable_missing)

In a nutshell, changes to the codebase are as follows:

Add data layer independent subclass of cerberus.Validator
- Support new signature of __init__ and validate
- Use _config-aware properties instead of bare member attributes to
  pass the resource, document_id and persisted_document to make
  them available to child validators
- Add schema-docstrings to all _validate_* methods
Adjust Mongo-specific Validator subclass
- Adjust _validate_type_* methods (following changes in Cerberus)
- Add schema-docstrings to all _validate_* methods
Add custom ErrorHandler to support VALIDATION_ERROR_AS_LIST
A few renames:
- ValidationError -> DocumentError
- propertyschema -> keyschema and keyschema -> valueschema
Adjust tests due to different validation error messages
(mostly for type)
Remove transparent_schema_rules without replacement
Remove default-handling, as Cerberus takes care of this now

dkellner · 2017-03-21T02:31:59Z

Great, thank you for re-opening this! I've quite busy with Eve-SQLAlchemy for now so did not have time myself to look into this again.

I just saw that setup.py and tox.ini still use dependency_links to install my development branch of Cerberus for testing - we should remove this. And AFAIK dependency link processing has been removed from pip.

nicolaiarocci · 2017-03-21T09:08:02Z

Excellent. Thank you @bcrochet for taking up the torch. @dkellner I'd appreciate a thoughtful review when you have time (use the Review feature).

bcrochet · 2017-03-22T13:22:18Z

@dkellner Removed the dependency_links lines from setup.py and tox.ini.

funkyfuture

i made a review from a cerberus-only pov. it's encouraging to see client code to be reduced. :-)

i also found two possible leverages for more:

cerberus could look up the rule constrains schemas from overridden methods when an overriding method doesn't provide one
cerberus could auto-create the properties that map to self._config keys or implement __getattr__ and __setattr__

i noted these.

funkyfuture · 2017-03-24T10:47:31Z

eve/validation.py

+        self._config['resource'] = value
+
+    @property
+    def document_id(self):


i think it's not a bad idea to add this property along with a schema_id to the vanilla Validator. essentially to have canonically named properties that extensions like error handlers can hook to.

Just to clarify: This is nothing concerning this very PR, right?

that's correct.

funkyfuture · 2017-03-24T10:53:58Z

eve/validation.py

+        self._config['persisted_document'] = value
+
+
+class DefaultErrorHandler(cerberus.errors.BasicErrorHandler):


assuming that config.VALIDATION_ERROR_AS_LIST doesn't change during runtime, you could reduce overhead with roughly this:

class SingleElementListMixin(object): def pretty_tree(self): ... if not config.VALIDATION_ERROR_AS_LIST: DefaultErrorHandler = type('DefaultErrorHandler', (cerberus.errors.BasicErrorHandler, SingleElementListMixin), {}) else: DefaultErrorHandler = cerberus.errors.BasicErrorHandler

I agree that we could do this check only one time at application startup. But I wonder if creating a dynamic type based on a config item can get confusing once someone will subclass this again.

How about renaming DefaultErrorHandler to something like SingleErrorsAsStringErrorHandler (or a better name if you can come with one) and then amend Validator.__init__ as follows?

if not config.VALIDATION_ERROR_AS_LIST: kwargs['error_handler'] = SingleErrorsAsStringErrorHandler

But again I cannot see a "real" issue with your (definitely more sophisticated) suggestion.

funkyfuture · 2017-03-24T10:55:16Z

eve/validation.py

+            return field not in self.document and \
+                self.persisted_document and \
+                field in self.persisted_document
+        return list(filter(persisted_but_not_in_document, fields))


aren't list comprehensions the common idioms for such expressions nowadays?

Agree, and it should make it more readable, too.

funkyfuture · 2017-03-24T11:05:53Z

eve/validation.py

+        """ {'type': ['dict', 'hashable', 'hashables']} """
+        persisted = self._filter_persisted_fields_not_in_document(dependencies)
+        if persisted:
+            dcopy = copy.copy(self.document)


would a deepcopy be less error prone here?

Yes, especially regarding the fact that the child validator can do normalization, too. I just looked into this and copy.copy was used in the original code as well. I think that's the reason I kept in there in the beginning. A "real" fix would include a test case where a plain copy fails so we never need to wonder about this again.

I'm leaving this for later. I'm not familiar enough with the code to be able to make an informed change here.

dkellner · 2017-03-25T05:21:42Z

@nicolaiarocci I will review this during the next days.

In the meantime we can discuss about updating the documentation (as already mentioned here: #917 (comment)). Additionally, we should add a section about upgrading, as this will introduce some breaking changes.

nicolaiarocci · 2017-03-25T08:01:15Z

Yes docs need to be amended and upgrading should be covered.

dkellner · 2017-03-28T15:46:42Z

I looked into this again, but this is my code without changes (so far) - so I'm not sure if I'm really the right person to review it ;). Nonetheless, I took some notes regarding needed documentation changes. The list is probably not complete, but a hopefully a good start:

_validate_type_uuid in custom_idfields.rst is wrong now (signature + return value)
same for _validate_type_objectid in validation.rst
still contains references to 'propertyschema' (should be 'keyschema' now),
this is probably true for other renames as well
remove references to TRANSPARENT_SCHEMA_RULES (config.rst, validation.rst)

@bcrochet Can you implement @funkyfuture's suggestions (amend to my commit if you want, I think that's fine in this case) and add another commit for the documentation changes?

bcrochet · 2017-03-28T22:05:27Z

@dkellner You're absolutely right. It is mostly your original patch. I think I maybe changed 2 lines, just to get it working with the latest cerebrus. But yes, I will take @funkyfuture 's suggestions and amend this commit.

funkyfuture · 2017-03-29T15:57:21Z

actually, i'm not sure whether this errorhandler is really needed. in cerberus the argument was that current client code should already be ready to deal with such structures as they were used for 2+ errors.

nicolaiarocci · 2017-05-03T07:14:26Z

@bcrochet is this work ready for merge, or you still have work left to do?

bcrochet · 2017-05-06T17:19:05Z

@nicolaiarocci I'm currently rebasing and will have some changes, so it's not quite ready.

bcrochet · 2017-05-06T19:20:07Z

I'd consider this ready. However, the python2.6 job is not passing, due to python2.6 being not found. Should this be dropped from tox.ini?

nicolaiarocci · 2017-05-07T14:02:04Z

Looks like the latest Travis update broke our .travis.yml. Can you please rebase (again) on top of master? Thanks!

bcrochet · 2017-05-07T22:53:19Z

Rebase done. I also went ahead and included doc changes.

nicolaiarocci · 2017-05-08T05:56:38Z

Excellent. One last effort and we're done. Please, update the 'breaking changes' section in the changelog. I guess a good reference would be Cerberus' own "upgrading to 1.0" page at https://cerberus.readthedocs.io/en/stable/upgrading.html.

(Based on original patch from dkellner) This is a rather big change. I still decided to do a single commit, as intermediate commits would be in a non-working state anyway. Breaking changes: - `keyschema` was renamed to `valueschema` and `propertyschema` to `keyschema` (following changes in Cerberus). - A PATCH on a document which misses a field having a default value will now result in setting this value, even if the field was not provided in the PATCH's payload. - Error messages for `keyschema` are now returned as dictionary. Before: {'propertyschema_dict': 'propertyschema_dict'} Now: {'keyschema_dict': {'AAA': "value does not match regex '[a-z]+'"}} - Error messages for `type` validations are different now (following changes in Cerberus). - It is no longer valid to have a field with `default` = None and `nullable` = False. (see patch.py:test_patch_nested_document_nullable_missing) In a nutshell, changes to the codebase are as follows: - Add data layer independent subclass of `cerberus.Validator` * Support new signature of `__init__` and `validate` * Use `_config`-aware properties instead of bare member attributes to pass the `resource`, `document_id` and `persisted_document` to make them available to child validators * Add schema-docstrings to all `_validate_*` methods - Adjust Mongo-specific `Validator` subclass * Adjust `_validate_type_*` methods (following changes in Cerberus) * Add schema-docstrings to all `_validate_*` methods - Add custom ErrorHandler to support `VALIDATION_ERROR_AS_LIST` - A few renames: * `ValidationError` -> `DocumentError` * `propertyschema` -> `keyschema` and `keyschema` -> `valueschema` - Adjust tests due to different validation error messages (mostly for `type`) - Remove `transparent_schema_rules` without replacement - Remove `default`-handling, as Cerberus takes care of this now

bcrochet · 2017-05-24T19:09:00Z

@nicolaiarocci Updated

nicolaiarocci · 2017-05-25T08:27:43Z

Merged after rebase,see 3bfff77.

Stellar, thank you.

nicolaiarocci added this to the 0.8 milestone Mar 21, 2017

nicolaiarocci mentioned this pull request Mar 21, 2017

Cerberus 1.0 Support #776

Closed

bcrochet force-pushed the cerberus-1.1 branch from 3920abe to 0c1eb89 Compare March 22, 2017 13:20

funkyfuture reviewed Mar 24, 2017

View reviewed changes

bcrochet force-pushed the cerberus-1.1 branch 3 times, most recently from 35e774f to 33c3415 Compare May 6, 2017 19:09

bcrochet force-pushed the cerberus-1.1 branch 2 times, most recently from b5e33b2 to 43cf038 Compare May 7, 2017 22:52

bcrochet force-pushed the cerberus-1.1 branch from 43cf038 to 1110f80 Compare May 18, 2017 01:08

nicolaiarocci closed this May 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for Cerberus 1.1 #1001

Support for Cerberus 1.1 #1001

bcrochet commented Mar 21, 2017

dkellner commented Mar 21, 2017

nicolaiarocci commented Mar 21, 2017

bcrochet commented Mar 22, 2017

funkyfuture left a comment •

edited

Loading

funkyfuture Mar 24, 2017

dkellner Mar 28, 2017

funkyfuture Mar 29, 2017

funkyfuture Mar 24, 2017 •

edited

Loading

dkellner Mar 28, 2017

funkyfuture Mar 24, 2017

dkellner Mar 28, 2017

funkyfuture Mar 24, 2017

dkellner Mar 28, 2017

bcrochet May 6, 2017

dkellner commented Mar 25, 2017

nicolaiarocci commented Mar 25, 2017

dkellner commented Mar 28, 2017

bcrochet commented Mar 28, 2017

funkyfuture commented Mar 29, 2017

nicolaiarocci commented May 3, 2017

bcrochet commented May 6, 2017

bcrochet commented May 6, 2017

nicolaiarocci commented May 7, 2017 •

edited

Loading

bcrochet commented May 7, 2017

nicolaiarocci commented May 8, 2017

bcrochet commented May 24, 2017

nicolaiarocci commented May 25, 2017

		self._config['persisted_document'] = value


		class DefaultErrorHandler(cerberus.errors.BasicErrorHandler):

Support for Cerberus 1.1 #1001

Support for Cerberus 1.1 #1001

Conversation

bcrochet commented Mar 21, 2017

dkellner commented Mar 21, 2017

nicolaiarocci commented Mar 21, 2017

bcrochet commented Mar 22, 2017

funkyfuture left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

funkyfuture Mar 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dkellner commented Mar 25, 2017

nicolaiarocci commented Mar 25, 2017

dkellner commented Mar 28, 2017

bcrochet commented Mar 28, 2017

funkyfuture commented Mar 29, 2017

nicolaiarocci commented May 3, 2017

bcrochet commented May 6, 2017

bcrochet commented May 6, 2017

nicolaiarocci commented May 7, 2017 • edited Loading

bcrochet commented May 7, 2017

nicolaiarocci commented May 8, 2017

bcrochet commented May 24, 2017

nicolaiarocci commented May 25, 2017

funkyfuture left a comment •

edited

Loading

funkyfuture Mar 24, 2017 •

edited

Loading

nicolaiarocci commented May 7, 2017 •

edited

Loading