Testing for "universal" assumptions

[harahu]

I just came across CrossHair, and, I must, say, it seems like a really interesting tool!

I have a use case that is related to the standard use case for CrossHair, but I am unsure if it actually is a good fit or not, and I hope someone can help me out in clearing it up. I've been using pandas in a lot of projects, but just got bitten by a bug resulting from the fact that pandas.Timedelta breaks what can be seen as a "universal" contract, that is, the Python equality-hash invariant:

Hashable objects which compare equal must have the same hash value.

This I call a "universal" contract because every Python object that has an __eq__() and a __hash__() method should adhere to this, except for some extremely good reason, and if they don't, it should be incredibly well documented.

Having been bit by this once, I am now asking myself: which other classes of object I use on a regular basis are breaking with fundamental assumptions like this? I'd really like a tool to figure this out for me.

My questions are then as follows:

1. Could one use CrossHair to test all classes used in a code base to challenge their adherence to the equality-hash invariant? Could this be done without the verbosity of having to specify the contract explicitly for all objects one are interested in? I am particularly concerned about this, since that would be a lot of work, and I regard it as safe to assume that this contract is intended, even if not specified. If CrossHair isn't the tool for the job, does anyone know about alternatives?
2. Does anyone know of other assumptions, invariants, protocols or contracts, in addition to the hash-equality invariant, that should be considered universal within Python, and that are really only dynamically testable?

[pschanely]

This is a really interesting topic.

First off, yes, CrossHair is a good tool for handling these kinds of problems. In particular, take a look at this example which uses a mixin to let classes opt-in to hash-eq consistency testing.

Now that's not quite what you're looking for: you want a tool to automatically apply this requirement.

One idea I've been toying with: an officially supported way to apply contracts to external classes that you don't control. With that kind of feature, you could apply a contract to object, I suppose, which would do what you want.

Indeed, one could imagine a variety of other universal contracts; maybe stuff like this?:

• __lt__ is irreflexive, antisymmetric, and transitive
• __add__ is associative (?)
• eval(x.__repr__()) == x (?)
• x.__copy__().__eq__(x)
• x.__copy__() is not x

[harahu]

This clears up a lot for me, so thanks!

First off, yes, CrossHair is a good tool for handling these kinds of problems. In particular, take a look at this example which uses a mixin to let classes opt-in to hash-eq consistency testing.

This is a neat trick and I like it. But as you point out, it does require that you control the code, which doesn't rule it out completely for 3rd party dependencies, but it gets close. It also means your class is inheriting from a Mixin for the sole purpose of stating that it adheres to something it should have adhered to in the first place.

One idea I've been toying with: an officially supported way to apply contracts to external classes that you don't control. With that kind of feature, you could apply a contract to object, I suppose, which would do what you want.

I think this is exactly what I had in mind with my question! For me that would be an amazing tool to have in my belt. At least if you have an extensive list of universal contracts you want to test for, which it seems one can come up with. Passing tests for such a list of universal contracts should probably be a low bar that potential dependencies need to pass before I'm willing to use them in production.

I can even imagine github readme badges where libraries brag about being tested for compliance with such a list, the same way they highlight adherence to a code style, or their test coverage.

Would love to test such a feature if you come around to implementing it!

Another item for the list, btw:

• __eq__ should be reflexive, symmetric and transitive

[pschanely]

You have some really great ideas in here. I am trying to wrap up some symbolic StringIO and BytesIO implementations, and will take a look at this soon after that.

Tangent: I'm also thinking about some other neat things we could do for library developers (see #145). Liking your idea of some kind of philosophical document & badge. Let me know if you'd like to contribute!

[harahu]

Let me know if you'd like to contribute!

Always interested in helping out, although CrossHair is quite far outside my professional domain, so unsure how effective I would be as a coder. But I can serve as a rubber duck when it comes to figuring out the api, should you need it.