Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating documentation for data format support #2086

Merged
merged 3 commits into from
Apr 20, 2021

Conversation

sameerz
Copy link
Collaborator

@sameerz sameerz commented Apr 7, 2021

This is a first pass at updating the documentation to warn users about unsupported data formats.

@andygrove if we have any more specific information about which formats are supported but can produce incorrect results for invalid inputs, it would be good for us to clarify that in the docs as well.

Fixes #2003

Signed-off-by: Sameer Raheja <sraheja@nvidia.com>
@sameerz sameerz added the documentation Improvements or additions to documentation label Apr 7, 2021
@sameerz sameerz added this to the Mar 29 - Apr 9 milestone Apr 7, 2021
@sameerz sameerz requested review from andygrove and jlowe April 7, 2021 01:33
@sameerz sameerz self-assigned this Apr 7, 2021
@andygrove
Copy link
Contributor

@sameerz I'm no longer sure which formats can produce incorrect results because there were a number of improvements since this documentation was originally written. However, @viadea did run into one example (I think) that led to this issue being filed.

@andygrove
Copy link
Contributor

@sameerz I'm no longer sure which formats can produce incorrect results because there were a number of improvements since this documentation was originally written. However, @viadea did run into one example (I think) that led to this issue being filed.

Actually, that isn't correct. @viadea was using an unsupported format, not a format that can produce incorrect results.

andygrove
andygrove previously approved these changes Apr 9, 2021
Copy link
Contributor

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sameerz
Copy link
Collaborator Author

sameerz commented Apr 9, 2021

@sameerz I'm no longer sure which formats can produce incorrect results because there were a number of improvements since this documentation was originally written. However, @viadea did run into one example (I think) that led to this issue being filed.

Actually, that isn't correct. @viadea was using an unsupported format, not a format that can produce incorrect results.

@andygrove is it still accurate to say that "some are supported but can produce incorrect results for invalid inputs" or should I update the document?

@sameerz sameerz marked this pull request as draft April 12, 2021 22:17
@sameerz
Copy link
Collaborator Author

sameerz commented Apr 12, 2021

@andygrove is going to investigate sql.incompatibleDateFormats and either remove it or provide more specific information about incompatible formats for the documentation. Putting this into a draft state for the time being.

sameerz added 2 commits April 20, 2021 03:56
Signed-off-by: Sameer Raheja <sraheja@nvidia.com>
@sameerz sameerz marked this pull request as ready for review April 20, 2021 05:02
@sameerz
Copy link
Collaborator Author

sameerz commented Apr 20, 2021

@andygrove please re-review

@sameerz
Copy link
Collaborator Author

sameerz commented Apr 20, 2021

build

1 similar comment
@pxLi
Copy link
Collaborator

pxLi commented Apr 20, 2021

build

Copy link
Contributor

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Sameer. I think this is much clearer now.

@sameerz sameerz merged commit cc3be0f into NVIDIA:branch-0.5 Apr 20, 2021
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
* Updating documentation for data format support

Signed-off-by: Sameer Raheja <sraheja@nvidia.com>

* Update config docs to reflect compatibility doc update

Signed-off-by: Sameer Raheja <sraheja@nvidia.com>
nartal1 pushed a commit to nartal1/spark-rapids that referenced this pull request Jun 9, 2021
* Updating documentation for data format support

Signed-off-by: Sameer Raheja <sraheja@nvidia.com>

* Update config docs to reflect compatibility doc update

Signed-off-by: Sameer Raheja <sraheja@nvidia.com>
@sameerz sameerz deleted the branch-0.5-doc-date-formats branch June 11, 2021 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Misleading documentation for spark.rapids.sql.incompatibleDateFormats.enabled
3 participants