Rework similary/duplicate query grouping

[living180]

The existing code for tracking and grouping similar and duplicate queries was complicated, hard to understand (nested dict comprehensions), and inefficient (multiple nested loops, using exceptions for flow control, calling key-computation functions twice for each query). Tweak the data structures to allow significant simplification of the code, and pull duplicated functionality into a shared function.

[matthiask]

I'm surprised. I thought there was a good reason for using the pprint module, because of unhashable types and other problems with data structures (I could be wrong, it has been a long time). LGTM.

[living180]

I thought there was a good reason for using the pprint module, because of unhashable types and other problems with data structures (I could be wrong, it has been a long time).

You are correct that raw_params cannot be used as a hash key directly because it can (and frequently does) include unhashable types such as lists. So the approach you used for duplicate_query_key() is to convert raw_params into a string (originally using pformat() in 87b7c31, then saferepr() in 48a825a). The approach is sound (and actually quite clever), but I think repr() should be sufficient for this case - it doesn't care about hashability and does give a useful string representation. The only additional protection that saferepr() brings is protection against recursive data structures (such as a list containing itself). However, trying to pass a recursive data structure as an SQL query parameter doesn't make sense and would cause things to blow up long before they reach the SQLPanel code, thus I don't think saferepr() is really necessary here.

[matthiask]

Thanks!