Support decimal, date, time, timestamp with time zone and timestamp python types in prepared statements and result sets

[mdesmet]

Fixes #150 , #32

[mdesmet]

As mentioned in #32, this change will impact the behavior on existing codebases. Should we provide a feature flag for this improved type detection?

[mdesmet]

Time with time zone (utc offset) doesn't really exist as a Python type. It's possible to have time with a timezone, but however without an actual moment in time, it's impossible to know the UTC offset. So I don't think this is something we can support for now.

[hashhar]

This introduces regression regarding range of values that can be represented and hence needs a toggle somewhere - maybe add an alternative Cursor implementation?

Left some other comments regarding tests.

[mdesmet]

To convert named timezones i have added pytz as dependency. As we support older python versions, it seems easier to just use pytz, which was already a test dependency. Maybe we should make this an optional dependency as it's only required for when the flag is activated. Please advice.

Starting from python 3.9 we could replace this with the standard zoneinfo (however seems still need an additional library on Windows: https://tzdata.readthedocs.io/en/latest/).

[hovaesco]

Maybe we should make this an optional dependency as it's only required for when the flag is activated.

Let's leave it as a required dependency since e.g I want to continue to use old python types and then switch to the new one.

[john-bodley]

Shouldn't this include timezone information?, i,.e., lines 537 and 539 are the same.

Note the dateutil.parser.parse method could be useful and replace the logic in lines 527–539, somewhat ignoring the specific raw_type, i.e., value should contain the timezone information if the raw_type is timestamp with time zone or time with time zone, i.e.,

from dateutil.parser import parse

if raw_type in ("time",  "time with time zone", "timestamp", "timestamp with time zone"):
    result = parse(value)
    
    if raw_type in ("time",  "time with time zone"):
        result = result.time().replace(tzinfo=result.tzinfo)
        
    return result

[mdesmet]

That's indeed a gap in the testing as well. On the library: currently we use pytz, and the format we will get is predictable so maybe no need to swap out the library.

[john-bodley]

Shouldn't this include timezone information?

Note the dateutil.parser.parse method could be useful and replace the logic in lines 527–539, somewhat ignoring the specific raw_type, i.e., value should contain the timezone information if the raw_type is timestamp with time zone or time with time zone, i.e.,

from dateutil.parser import parse

if raw_type in ("time",  "time with time zone", "timestamp", "timestamp with time zone"):
    result = parse(value)
    
    if raw_type in ("time",  "time with time zone"):
        result = result.time().replace(tzinfo=result.tzinfo)
        
    return result

[hashhar]

A few remaining comments. Thanks for the work on this @mdesmet.

Looks almost good to go.

[hashhar]

LGTM % last batch of comments.

[hashhar]

@nineinchnick / @hovaesco Can you please double check this? It looks correct to me - not sure of simpler alternatives.

[hovaesco]

LGTM, we could avoid checking if matches.group(2) == '+' and just add +/- and don't have a conditional here. But I don't have a strong opinion about either way.

[hashhar]

Good point. It seems for values like 2022-01-01 01:01:01 05:30 we'd convert this to negative offset which seems wrong.

@mdesmet can you passthrough the sign here? Or explicitly compare to - as well. the default should be positive offset AFAIK.

[mdesmet]

Good point about the default. Will make that change now.

[hashhar]

LGTM % a few more comments @mdesmet

#160 (comment) and #160 (comment)

[hashhar]

Thank you.