Checking pandas column holding datetime.time class values

[HerveMignot]

Having, in a pandas.DataFrame, a column containing durations encoded as datetime.time, with some null values (coded as pd.NaT, below being a sample of results when applying type() to cells in the column):

<class 'datetime.time'>                          92304
<class 'pandas._libs.tslibs.nattype.NaTType'>    20677
Name: flightDuration, dtype: int64

is there a Pandera Data Type that can be used to validate such a column (nullable=True)?

Thank you.

[cosmicBboy]

hi @HerveMignot, have you tried using the datetime.time as the column datatype, like pa.Column(datetime.time). Pandera will interpret this as an object, so I think any null representation would work.

Do you have a reproducible code snippet example that you're working with?

[HerveMignot]

Hi, thank you for the input.

I just tried with a pa.Column(datetime.time, nullable=True) but Ì got when defining the pa.DataFrameSchema:

TypeError: dtype '<class 'datetime.time'>' not understood

I prepared a code snippet to give some context:

example_df = (pd.DataFrame(data={'Duration_Raw': ['2:10:00', '1:10:00', '0:50:00', '', '7:10:00']})
                .assign(flightDuration=lambda _df: pd.to_datetime(_df['Duration_Raw'], format='%H:%M:%S', errors='coerce').dt.time)
                [['flightDuration']])

example_pa_schema = pa.DataFrameSchema({
    'flightDuration': pa.Column(datetime.time, nullable=True),
    }, strict=True, ordered=True)

I'll try other way to specify a datetime.time dtype.