Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement serialization/de-serialization support for chrono::Duration #4037

Merged
merged 10 commits into from
May 24, 2024
89 changes: 86 additions & 3 deletions diesel/src/pg/types/date_and_time/chrono.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
extern crate chrono;
use self::chrono::{DateTime, Duration, Local, NaiveDate, NaiveDateTime, NaiveTime, TimeZone, Utc};

use super::{PgDate, PgTime, PgTimestamp};
use super::{PgDate, PgInterval, PgTime, PgTimestamp};
use crate::deserialize::{self, FromSql};
use crate::pg::{Pg, PgValue};
use crate::serialize::{self, Output, ToSql};
use crate::sql_types::{Date, Time, Timestamp, Timestamptz};
use crate::sql_types::{Date, Interval, Time, Timestamp, Timestamptz};

// Postgres timestamps start from January 1st 2000.
fn pg_epoch() -> NaiveDateTime {
Expand Down Expand Up @@ -139,6 +139,48 @@ impl FromSql<Date, Pg> for NaiveDate {
}
}

const DAYS_PER_MONTH: i32 = 30;
Copy link
Member

@Ten0 Ten0 May 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that seems arbitrary (although it's probably the most sensible value if there has to be any)

Since we already have a type that represents pg's interval, maybe what we'd want instead is to propose typed conversion functions from that to chrono, which would make very clear that there's this going on and that the roundtrip is not expected to preserve the values and that adding the Rust types is not expected to produce the same results as adding on the pg side?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please read line 175~178. This is a ratio when the Postgres explicitly converts between months and days.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree these are the most sensible values.

What I mean is that '2024-05-25'::Timestamp + '1 month'::interval != DateTime::from_sql('2024-05-25'::Timestamp) + chrono::TimeDelta::from_sql('1 month'::interval), and ToSql(chrono::TimeDelta::from_sql('1 month'::interval)) != '1 month'::interval, and I'm wondering if this behavior doesn't make for enough of a trap that we should instead have users deserialize to PgInterval and just provide functions that make conversions from that to chrono::TimeDelta reasonably easy but more explicit on that it may lose information and behave differently.

Copy link
Contributor Author

@McDic McDic May 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know there isn't any separated dimension or field for months in chrono::Duration. I don't think any other widely used Rust "timedelta" types support it. It is fundamental behavior difference between PostgreSQL interval and widely used (and standard in my opinion) interval data types.

PostgreSQL returns timedelta with "0 month" when it is produced by datetime - datetime. For example SELECT '2024-06-26'::Timestamp - '2024-05-26'::Timestamp will return '31 days'::interval instead of '1 month'::interval. The month field is used only when the user explicitly specifies it(would be specifying SQL literals for diesel users), so the error case you mentioned occurs on only who uses "month" dimension in their SQL query explicitly.

I am not sure if there would be many cases which uses month field explicitly, as PostgreSQL also warns these special behaviors based on political stuffs like daylight savings in their official documentation. In my opinion all users who use "month" dimension in interval should know their different behaviors.

Also this PR does not delete usage of PgInterval, so some sensitive users may still use it if they want. However I agree it makes sense to document there could be some information loss.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@McDic Could you open a PR for the documentation clarification?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weiznich Sure. I will create a PR. Is this urgent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #4051 !

const SECONDS_PER_DAY: i64 = 60 * 60 * 24;
const MICROSECONDS_PER_SECOND: i64 = 1_000_000;

#[cfg(all(feature = "chrono", feature = "postgres_backend"))]
impl ToSql<Interval, Pg> for Duration {
fn to_sql<'b>(&'b self, out: &mut Output<'b, '_, Pg>) -> serialize::Result {
let microseconds: i64 = if let Some(v) = self.num_microseconds() {
v % (MICROSECONDS_PER_SECOND * SECONDS_PER_DAY)
} else {
return Err("Failed to create microseconds by overflow".into());
};
let days: i32 = self
.num_days()
.try_into()
.expect("Failed to get i32 days from i64");
// We don't use months here, because in PostgreSQL
// `timestamp - timestamp` returns interval where
// every delta is contained in days and microseconds, and 0 months.
// https://www.postgresql.org/docs/current/functions-datetime.html
let interval = PgInterval {
microseconds,
days,
months: 0,
};
<PgInterval as ToSql<Interval, Pg>>::to_sql(&interval, &mut out.reborrow())
}
}

#[cfg(all(feature = "chrono", feature = "postgres_backend"))]
impl FromSql<Interval, Pg> for Duration {
fn from_sql(bytes: PgValue<'_>) -> deserialize::Result<Self> {
let interval: PgInterval = FromSql::<Interval, Pg>::from_sql(bytes)?;
// We use 1 month = 30 days and 1 day = 24 hours, as postgres
// use those ratios as default when explicitly converted.
// For reference, please read `justify_interval` from this page.
// https://www.postgresql.org/docs/current/functions-datetime.html
let days = interval.months * DAYS_PER_MONTH + interval.days;
Ok(Duration::days(days as i64) + Duration::microseconds(interval.microseconds))
}
}

#[cfg(test)]
mod tests {
extern crate chrono;
Expand All @@ -149,7 +191,7 @@ mod tests {
use crate::dsl::{now, sql};
use crate::prelude::*;
use crate::select;
use crate::sql_types::{Date, Time, Timestamp, Timestamptz};
use crate::sql_types::{Date, Interval, Time, Timestamp, Timestamptz};
use crate::test_helpers::connection;

#[test]
Expand Down Expand Up @@ -331,4 +373,45 @@ mod tests {
query.get_result::<NaiveDate>(connection)
);
}

/// Get test duration and corresponding literal SQL strings.
fn get_test_duration_and_literal_strings() -> (Duration, Vec<&'static str>) {
(
Duration::days(60) + Duration::minutes(1) + Duration::microseconds(123456),
vec![
"60 days 1 minute 123456 microseconds",
"2 months 1 minute 123456 microseconds",
"5184060 seconds 123456 microseconds",
"60 days 60123456 microseconds",
"59 days 24 hours 60.123456 seconds",
"60 0:01:00.123456",
"58 48:01:00.123456",
"P0Y2M0DT0H1M0.123456S",
"0-2 0:01:00.123456",
"P0000-02-00T00:01:00.123456",
"1440:01:00.123456",
"1 month 30 days 0.5 minutes 30.123456 seconds",
],
)
}

#[test]
fn duration_encode_correctly() {
let connection = &mut connection();
let (duration, literal_strings) = get_test_duration_and_literal_strings();
for literal in literal_strings {
let query = select(sql::<Interval>(&format!("'{}'::interval", literal)).eq(duration));
assert!(query.get_result::<bool>(connection).unwrap());
}
}

#[test]
fn duration_decode_correctly() {
let connection = &mut connection();
let (duration, literal_strings) = get_test_duration_and_literal_strings();
for literal in literal_strings {
let query = select(sql::<Interval>(&format!("'{}'::interval", literal)));
assert_eq!(Ok(duration), query.get_result::<Duration>(connection));
}
}
}
3 changes: 3 additions & 0 deletions diesel/src/sql_types/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -293,13 +293,16 @@ pub struct Date;
/// ### [`ToSql`](crate::serialize::ToSql) impls
///
/// - [`PgInterval`] which can be constructed using [`IntervalDsl`]
/// - [`chrono::Duration`][Duration] with `feature = "chrono"`
///
/// ### [`FromSql`](crate::deserialize::FromSql) impls
///
/// - [`PgInterval`] which can be constructed using [`IntervalDsl`]
/// - [`chrono::Duration`][Duration] with `feature = "chrono"`
///
/// [`PgInterval`]: ../pg/data_types/struct.PgInterval.html
/// [`IntervalDsl`]: ../pg/expression/extensions/trait.IntervalDsl.html
/// [Duration]: https://docs.rs/chrono/*/chrono/type.Duration.html
#[derive(Debug, Clone, Copy, Default, QueryId, SqlType)]
#[diesel(postgres_type(oid = 1186, array_oid = 1187))]
pub struct Interval;
Expand Down
7 changes: 6 additions & 1 deletion diesel/src/type_impls/date_and_time.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ mod chrono {
use self::chrono::*;
use crate::deserialize::FromSqlRow;
use crate::expression::AsExpression;
use crate::sql_types::{Date, Time, Timestamp};
use crate::sql_types::{Date, Interval, Time, Timestamp};

#[derive(AsExpression, FromSqlRow)]
#[diesel(foreign_derive)]
Expand Down Expand Up @@ -49,6 +49,11 @@ mod chrono {
)]
#[cfg_attr(feature = "sqlite", diesel(sql_type = crate::sql_types::TimestamptzSqlite))]
struct DateTimeProxy<Tz: TimeZone>(DateTime<Tz>);

#[derive(AsExpression, FromSqlRow)]
#[diesel(foreign_derive)]
#[diesel(sql_type = Interval)]
struct DurationProxy(Duration);
}

#[cfg(feature = "time")]
Expand Down
Loading