Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sum of cardinality for array column verifier results #13916

Merged
merged 2 commits into from
Jan 7, 2020

Conversation

ShenYi
Copy link
Contributor

@ShenYi ShenYi commented Jan 2, 2020

For array column mismatch, this change adds additional information
of the total number of elements across all rows for the same column.

Part of #13809

== RELEASE NOTES ==

Verifier Changes
* Add checks for the cardinalities sum when validating an array column.

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jan 2, 2020

CLA Check
The committers are authorized under a signed CLA.

  • ✅ shenyi (787a6319ff6fcd40a3665d57e4e399c554fba610)

Copy link
Contributor

@mbasmanova mbasmanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but let's have @caithagoras take a look as well.

@mbasmanova
Copy link
Contributor

@ShenYi would you sign EasyCLA?

@ShenYi
Copy link
Contributor Author

ShenYi commented Jan 3, 2020

@ShenYi would you sign EasyCLA?

I tried with corporate Facebook, got message "you are not whitelisted...". I have sent a message through the "Contact CLA Manager" link on the same page.

@mbasmanova
Copy link
Contributor

@ShenYi would you sign EasyCLA?

I tried with corporate Facebook, got message "you are not whitelisted...". I have sent a message through the "Contact CLA Manager" link on the same page.

@aweisberg Ariel, is this something you could help with?

Copy link
Contributor

@caithagoras caithagoras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually, this PR does 2 things.

  • Repurpose OrderableArrayColumnValidator as ArrayColumnValidator
  • Add array cardinality in ArrayColumnValidator.

For readability purpose, it would be better to separate it in to 2 commits. You can use interactive rebase to edit multiple commits. In case you're not familiar with this command, here is a reference: https://thoughtbot.com/blog/git-interactive-rebase-squash-amend-rewriting-history

@caithagoras
Copy link
Contributor

@ShenYi @mbasmanova I can see EasyCLA green now.

@caithagoras
Copy link
Contributor

@ShenYi Release notes:
"* Add checks for the cardinalities sum when validating an array column."

Copy link
Contributor

@caithagoras caithagoras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm % nit

@caithagoras
Copy link
Contributor

Also, Travis seems to be broken, although not relevant to this PR.

@caithagoras
Copy link
Contributor

Please ping on the PR when the comments are addressed, and @mbasmanova will merge for you.

@ShenYi
Copy link
Contributor Author

ShenYi commented Jan 3, 2020

@mbasmanova Addressed all the comments.

@mbasmanova
Copy link
Contributor

@ShenYi Ping me when Travis is green and I'll merge.

@caithagoras
Copy link
Contributor

@ShenYi Release notes are only for changes that users / customers sees and cares about, so you can remove the first item, which is a refactoring.

@ShenYi ShenYi closed this Jan 4, 2020
@ShenYi ShenYi reopened this Jan 4, 2020
@ShenYi ShenYi closed this Jan 4, 2020
@ShenYi ShenYi reopened this Jan 4, 2020
@ShenYi ShenYi closed this Jan 6, 2020
@ShenYi ShenYi reopened this Jan 6, 2020
@mbasmanova
Copy link
Contributor

@ShenYi I'm seeing this PR closing and reopening again and again. What's causing this?

Refactor OrderableArrayColumnValidator into ArrayColumnValidator.
Add logic to also support non-orderable array types.
For array column mismatch, add additional information of the total
number of elements across all rows.
@ShenYi
Copy link
Contributor Author

ShenYi commented Jan 7, 2020

@mbasmanova test passed and the change is ready to be merged now.

One of the test kept failing due to a bad maven zip from repository during the initial build. All subsequent builds load the same corrupted package from travis cache. Build passed after I manually deleted the maven dependencies for the pull request.

@mbasmanova mbasmanova merged commit 176e8f8 into prestodb:master Jan 7, 2020
@mbasmanova
Copy link
Contributor

@ShenYi Thank you for the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants