Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support COUNT(DISTINCT timestamps) #319

Merged
merged 3 commits into from
May 12, 2021

Conversation

charlibot
Copy link
Contributor

Which issue does this PR close?

Closes #314 .

What changes are included in this PR?

Handles DataType::Timestamp in ScalarValue::List with a custom macro build_timestamp_list that takes the time unit and timezone.

Are there any user-facing changes?

@charlibot
Copy link
Contributor Author

Hi! I'm pretty new to rust and this project so might have missed it but are there any tests I should add to?

@Dandandan
Copy link
Contributor

Dandandan commented May 11, 2021

Hey @charlibot thanks for opening this PR - awesome.
I think it would be nice to have at least a simple test for running a COUNT DISTINCT (timestamp) even if it is a simple

SELECT COUNT(DISTINCT (CAST('2018-11-13T17:11:10.011' as timestamp)));

Or with some more values to check whether the number of distinct values are calculated correctly.

Currently we have some end-to-end tests in datafusion/tests/sql.rs - do you feel like adding a test there?

@charlibot
Copy link
Contributor Author

Hey @Dandandan, thanks for the pointer. I'll try to add that in tomorrow

@alamb alamb changed the title Pattern match on DataType::Timestamp and build the list Support COUNT(DISTINCT timestamps) May 12, 2021
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great to me. Thank you @charlibot and thanks for adding the test.

@@ -1959,6 +1959,19 @@ async fn to_timestamp() -> Result<()> {
Ok(())
}

#[tokio::test]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Copy link
Contributor

@Dandandan Dandandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great @charlibot ! Thanks 💯

@Dandandan Dandandan merged commit cdc8b2d into apache:master May 12, 2021
@houqp houqp added datafusion Changes in the datafusion crate enhancement New feature or request labels Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error while running COUNT DISTINCT (timestamp): 'Unexpected DataType for list
4 participants